Compiling ISSM on Ubuntu 16.04.5

iisola97

I was trying to compile ISSM on Ubuntu following the steps provided at: https://issm.jpl.nasa.gov/download/unix/, but I'm having the following issues:

./cshrc doesn't seem to work because terminal doesn't recognize the setenv command. These are the error messages:
No command 'setenv' found, did you mean:
Command 'netenv' from package 'netenv' (universe)
setenv: command not found
No command 'setenv' found, did you mean:
Command 'netenv' from package 'netenv' (universe)
setenv: command not found
I don't fully understand how to install the mandatory external packages. I was trying to follow a previous post on this forum named 'Compiling with Ubuntu with Python provided by apt' (https://issm.ess.uci.edu/forum/d/147-compiling-with-ubuntu-with-python-provided-by-apt) but I didn't have luck, I'm getting some errors.
Finally, I created my configure.sh file but when I run 'autoreconf -ivf' on terminal I'm getting the following error:
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force -I m4
autoreconf: configure.ac: tracing
autoreconf: configure.ac: not using Libtool
autoreconf: running: /usr/bin/autoconf --force
autoreconf: running: /usr/bin/autoheader --force
autoreconf: running: automake --add-missing --copy --force-missing
src/c/Makefile.am:9: error: Libtool library used but 'LIBTOOL' is undefined
src/c/Makefile.am:9: The usual way to define 'LIBTOOL' is to add 'LT_INIT'
src/c/Makefile.am:9: to 'configure.ac' and run 'aclocal' and 'autoconf' again.
src/c/Makefile.am:9: If 'LT_INIT' is in 'configure.ac', make sure
src/c/Makefile.am:9: its definition is in aclocal's search path.
src/mobile/native/Makefile.am:2: error: Libtool library used but 'LIBTOOL' is undefined
src/mobile/native/Makefile.am:2: The usual way to define 'LIBTOOL' is to add 'LT_INIT'
src/mobile/native/Makefile.am:2: to 'configure.ac' and run 'aclocal' and 'autoconf' again.
src/mobile/native/Makefile.am:2: If 'LT_INIT' is in 'configure.ac', make sure
src/mobile/native/Makefile.am:2: its definition is in aclocal's search path.
src/wrappers/javascript/Makefile.am:26: error: Libtool library used but 'LIBTOOL' is undefined
src/wrappers/javascript/Makefile.am:26: The usual way to define 'LIBTOOL' is to add 'LT_INIT'
src/wrappers/javascript/Makefile.am:26: to 'configure.ac' and run 'aclocal' and 'autoconf' again.
src/wrappers/javascript/Makefile.am:26: If 'LT_INIT' is in 'configure.ac', make sure
src/wrappers/javascript/Makefile.am:26: its definition is in aclocal's search path.
src/wrappers/javascript/Makefile.am:29: error: Libtool library used but 'LIBTOOL' is undefined
src/wrappers/javascript/Makefile.am:29: The usual way to define 'LIBTOOL' is to add 'LT_INIT'
src/wrappers/javascript/Makefile.am:29: to 'configure.ac' and run 'aclocal' and 'autoconf' again.
src/wrappers/javascript/Makefile.am:29: If 'LT_INIT' is in 'configure.ac', make sure
src/wrappers/javascript/Makefile.am:29: its definition is in aclocal's search path.
src/wrappers/matlab/Makefile.am:11: error: Libtool library used but 'LIBTOOL' is undefined
src/wrappers/matlab/Makefile.am:11: The usual way to define 'LIBTOOL' is to add 'LT_INIT'
src/wrappers/matlab/Makefile.am:11: to 'configure.ac' and run 'aclocal' and 'autoconf' again.
src/wrappers/matlab/Makefile.am:11: If 'LT_INIT' is in 'configure.ac', make sure
src/wrappers/matlab/Makefile.am:11: its definition is in aclocal's search path.
src/wrappers/matlab/Makefile.am:14: error: Libtool library used but 'LIBTOOL' is undefined
src/wrappers/matlab/Makefile.am:14: The usual way to define 'LIBTOOL' is to add 'LT_INIT'
src/wrappers/matlab/Makefile.am:14: to 'configure.ac' and run 'aclocal' and 'autoconf' again.
src/wrappers/matlab/Makefile.am:14: If 'LT_INIT' is in 'configure.ac', make sure
src/wrappers/matlab/Makefile.am:14: its definition is in aclocal's search path.
src/wrappers/python/Makefile.am:10: error: Libtool library used but 'LIBTOOL' is undefined
src/wrappers/python/Makefile.am:10: The usual way to define 'LIBTOOL' is to add 'LT_INIT'
src/wrappers/python/Makefile.am:10: to 'configure.ac' and run 'aclocal' and 'autoconf' again.
src/wrappers/python/Makefile.am:10: If 'LT_INIT' is in 'configure.ac', make sure
src/wrappers/python/Makefile.am:10: its definition is in aclocal's search path.
autoreconf: automake failed with exit status: 1

LT_INIT is in 'configure.ac', but I don't know how to 'add its definition in aclocal's search path'.
I don't fully understand if configure.sh file is useful or not.

Overall, I'm not very familiar with terminal coding. I don't fully understand how to follow the compiling steps correctly and which lines of code should I run in the terminal. I would appreciate any help.

Thanks,

Ignacio

justinquinn

Hi Ignacio,
(1) Can you try using _/.bashrc instead? I personally have only used the bash interpreter for our builds, so I cannot confirm that csh will work.
(2) Take a look at $ISSM_DIR/jenkins/linux64_ross-python. This is the configuration script we use for our build system, and I can guarantee that this selection of packages and configuration will work under Ubuntu 16.04.
(3) It looks like you may not have run source etc/environment.sh after installing autotools. Sourcing this script after each successful external package installation is important as it modifies the PATH variable to include the new package.

Lastly, a general roadmap is the following,
(1) Open the configuration file listed above. For each of the listed external packages, and in that order, cd $ISSM_DIR/externalpackages/<pkg_name>, where <pkg_name> is the name of the pacakage. Then, ./<install_script>, where <install_script> is the name of the installation script corresponding to the external package in the aforementioned configuration script.
(2) If the package successfully installs, cd $ISSM_DIR; source etc/environment.sh. Otherwise, cat compil.log to get an idea of what might be going wrong (feel free to post here again).
(3) Copy the value assigned to ISSM_CONFIG from the aforementioned configuration file to your configure.sh. Then, cd $ISSM_DIR; autoreconf -ivf; ./configure.sh.
(4) Finally, make; make install.

Please let us know if you run into any other issues!

iisola97

Hi Justin,

Thanks for the response.

Here are some updates:

I'm only using .bashrc now.
I took a look at $ISSM_DIR/jenkins/linux64_ross-python. The only thing I changed from this file was the MATLAB path to match mine.
I followed the general roadmap you provided me with. I didn't have any issues when installing most of the external packages but I came across with an issue while installing petsc.

UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details):

Unable to download FBLASLAPACK
file could not be opened successfully
Downloaded package FBLASLAPACK from: http://ftp.mcs.anl.gov/pub/petsc/externalpackages/fblaslapack-3.4.2.tar.gz is not a tarball.
[or installed python cannot process compressed files]

If you are behind a firewall - please fix your proxy and rerun ./configure
For example at LANL you may need to set the environmental variable http_proxy (or HTTP_PROXY?) to http://proxyout.lanl.gov
You can run with --with-packages-dir=/adirectory and ./configure will instruct you what packages to download manually
or you can download the above URL manually, to /yourselectedlocation/fblaslapack-3.4.2.tar.gz
and use the configure option:
--download-fblaslapack=/yourselectedlocation/fblaslapack-3.4.2.tar.gz
file could not be opened successfully
Downloaded package FBLASLAPACK from: ftp://ftp.mcs.anl.gov/pub/petsc/externalpackages/fblaslapack-3.4.2.tar.gz is not a tarball.
[or installed python cannot process compressed files]
If you are behind a firewall - please fix your proxy and rerun ./configure
For example at LANL you may need to set the environmental variable http_proxy (or HTTP_PROXY?) to http://proxyout.lanl.gov
You can run with --with-packages-dir=/adirectory and ./configure will instruct you what packages to download manually
or you can download the above URL manually, to /yourselectedlocation/fblaslapack-3.4.2.tar.gz
and use the configure option:
--download-fblaslapack=/yourselectedlocation/fblaslapack-3.4.2.tar.gz
*******************************************************************************

I downloaded that file and saved it at $ISSM_DIR/externalpackages/petsc, and added to install-3.7-linux64.sh the last line of the configure section:

#!/bin/bash
set -eu

#Some cleanup
rm -rf install petsc-3.7.6 src
mkdir install src

#Download from ISSM server
$ISSM_DIR/scripts/DownloadExternalPackage.sh 'http://issm.jpl.nasa.gov/files/externalpackages/petsc-lite-3.7.6.tar.gz' 'petsc-3.7.6.tar.gz'

#Untar and move petsc to install directory
tar -zxvf petsc-3.7.6.tar.gz
mv petsc-3.7.6/* src/
rm -rf petsc-3.7.6

#configure
cd src
./config/configure.py \
--prefix="$ISSM_DIR/externalpackages/petsc/install" \
--with-mpi-dir="$ISSM_DIR/externalpackages/mpich/install" \
--PETSC_DIR="$ISSM_DIR/externalpackages/petsc/src" \
--with-debugging=0 \
--with-valgrind=0 \
--with-x=0 \
--with-ssl=0 \
--with-shared-libraries=1 \
--download-metis=1 \
--download-parmetis=1 \
--download-mumps=1 \
--download-scalapack=1 \
--download-fblaslapack=1 \
--with-pic=1 \
--download-fblaslapack=$ISSM_DIR/externalpackages/petsc/fblaslapack-3.4.2.tar.gz

#Compile and intall
make
make install

However, I tried installing petsc again and this message showed up:

UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details):

Unable to download FBLASLAPACK
file could not be opened successfully
Downloaded package FBLASLAPACK from: file:///home/ignacio/research/mars/ISSM_source/trunk/externalpackages/petsc/fblaslapack-3.4.2.tar.gz is not a tarball.
[or installed python cannot process compressed files]

If you are behind a firewall - please fix your proxy and rerun ./configure
For example at LANL you may need to set the environmental variable http_proxy (or HTTP_PROXY?) to http://proxyout.lanl.gov
You can run with --with-packages-dir=/adirectory and ./configure will instruct you what packages to download manually
or you can download the above URL manually, to /yourselectedlocation/fblaslapack-3.4.2.tar.gz
and use the configure option:
--download-fblaslapack=/yourselectedlocation/fblaslapack-3.4.2.tar.gz
*******************************************************************************
Although this package couldn't be installed properly (I think), I continued installing the rest of the external packages with the order specified and their names of installation script. I sourced cd $ISSM_DIR; source etc/environment.sh every time after I installed a package. However, when I ran the very last step cd $ISSM_DIR; autoreconf -ivf; ./configure.sh, I got the same error as before:

autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force -I m4
autoreconf: configure.ac: tracing
autoreconf: configure.ac: not using Libtool
autoreconf: running: /usr/bin/autoconf --force
autoreconf: running: /usr/bin/autoheader --force
autoreconf: running: automake --add-missing --copy --force-missing
src/c/Makefile.am:9: error: Libtool library used but 'LIBTOOL' is undefined
src/c/Makefile.am:9: The usual way to define 'LIBTOOL' is to add 'LT_INIT'
src/c/Makefile.am:9: to 'configure.ac' and run 'aclocal' and 'autoconf' again.
src/c/Makefile.am:9: If 'LT_INIT' is in 'configure.ac', make sure
src/c/Makefile.am:9: its definition is in aclocal's search path.
src/mobile/native/Makefile.am:2: error: Libtool library used but 'LIBTOOL' is undefined
src/mobile/native/Makefile.am:2: The usual way to define 'LIBTOOL' is to add 'LT_INIT'
src/mobile/native/Makefile.am:2: to 'configure.ac' and run 'aclocal' and 'autoconf' again.
src/mobile/native/Makefile.am:2: If 'LT_INIT' is in 'configure.ac', make sure
src/mobile/native/Makefile.am:2: its definition is in aclocal's search path.
src/wrappers/javascript/Makefile.am:26: error: Libtool library used but 'LIBTOOL' is undefined
src/wrappers/javascript/Makefile.am:26: The usual way to define 'LIBTOOL' is to add 'LT_INIT'
src/wrappers/javascript/Makefile.am:26: to 'configure.ac' and run 'aclocal' and 'autoconf' again.
src/wrappers/javascript/Makefile.am:26: If 'LT_INIT' is in 'configure.ac', make sure
src/wrappers/javascript/Makefile.am:26: its definition is in aclocal's search path.
src/wrappers/javascript/Makefile.am:29: error: Libtool library used but 'LIBTOOL' is undefined
src/wrappers/javascript/Makefile.am:29: The usual way to define 'LIBTOOL' is to add 'LT_INIT'
src/wrappers/javascript/Makefile.am:29: to 'configure.ac' and run 'aclocal' and 'autoconf' again.
src/wrappers/javascript/Makefile.am:29: If 'LT_INIT' is in 'configure.ac', make sure
src/wrappers/javascript/Makefile.am:29: its definition is in aclocal's search path.
src/wrappers/matlab/Makefile.am:11: error: Libtool library used but 'LIBTOOL' is undefined
src/wrappers/matlab/Makefile.am:11: The usual way to define 'LIBTOOL' is to add 'LT_INIT'
src/wrappers/matlab/Makefile.am:11: to 'configure.ac' and run 'aclocal' and 'autoconf' again.
src/wrappers/matlab/Makefile.am:11: If 'LT_INIT' is in 'configure.ac', make sure
src/wrappers/matlab/Makefile.am:11: its definition is in aclocal's search path.
src/wrappers/matlab/Makefile.am:14: error: Libtool library used but 'LIBTOOL' is undefined
src/wrappers/matlab/Makefile.am:14: The usual way to define 'LIBTOOL' is to add 'LT_INIT'
src/wrappers/matlab/Makefile.am:14: to 'configure.ac' and run 'aclocal' and 'autoconf' again.
src/wrappers/matlab/Makefile.am:14: If 'LT_INIT' is in 'configure.ac', make sure
src/wrappers/matlab/Makefile.am:14: its definition is in aclocal's search path.
src/wrappers/python/Makefile.am:10: error: Libtool library used but 'LIBTOOL' is undefined
src/wrappers/python/Makefile.am:10: The usual way to define 'LIBTOOL' is to add 'LT_INIT'
src/wrappers/python/Makefile.am:10: to 'configure.ac' and run 'aclocal' and 'autoconf' again.
src/wrappers/python/Makefile.am:10: If 'LT_INIT' is in 'configure.ac', make sure
src/wrappers/python/Makefile.am:10: its definition is in aclocal's search path.
autoreconf: automake failed with exit status: 1
bash: ./configure.sh: Permission denied

Just in case, I'm pasting configure.sh and configure.ac:

configure.sh

#MATLAB path
MATLAB_PATH="_{isaac/bin/MATLAB/R2018b/bin"}

#ISSM CONFIGURATION
ISSM_CONFIG='--prefix=$ISSM_DIR \
--disable-static \
--with-matlab-dir=$MATLAB_PATH \
--with-python-dir=/usr \
--with-python-numpy-dir=/home/jenkins/.local/lib/python2.7/site-packages/numpy \
--with-triangle-dir=$ISSM_DIR/externalpackages/triangle/install \
--with-mpi-include=$ISSM_DIR/externalpackages/mpich/install/include \
--with-mpi-libflags="-L$ISSM_DIR/externalpackages/mpich/install/lib -lmpi -lmpicxx -lmpifort" \
--with-petsc-dir=$ISSM_DIR/externalpackages/petsc/install \
--with-metis-dir=$ISSM_DIR/externalpackages/petsc/install \
--with-mumps-dir=$ISSM_DIR/externalpackages/petsc/install \
--with-scalapack-dir=$ISSM_DIR/externalpackages/petsc/install \
--with-blas-lapack-dir=$ISSM_DIR/externalpackages/petsc/install \
--with-chaco-dir="$ISSM_DIR/externalpackages/chaco/install" \
--with-fortran-lib="-L/usr/lib/gcc/x86_64-linux-gnu/4.9 -lgfortran" \
--with-m1qn3-dir=$ISSM_DIR/externalpackages/m1qn3/install \
--with-numthreads=4 \
--enable-development \
--enable-debugging '

configure.ac

Process this file with autoconf to produce a configure script.

#AUTOCONF
AC_INIT([Ice Sheet System Model (ISSM)],[4.15],[issm@jpl.nasa.gov],[issm],[http://issm.jpl.nasa.gov]) #Initializing configure
AC_CONFIG_AUX_DIR([./aux-config]) #Put config files in aux-config
AC_CONFIG_MACRO_DIR([m4]) #m4 macros are located in m4
m4_include([m4/issm_options.m4])

#print header
AC_MSG_NOTICE(============================================================================)
AC_MSG_NOTICE(= AC_PACKAGE_NAME AC_PACKAGE_VERSION configuration =)
AC_MSG_NOTICE(============================================================================)

#Determine System type and OS
AC_CANONICAL_TARGET

#Compilers
AC_PROG_CC([icc cl icl gcc])
AC_PROG_CPP
AC_PROG_CXX([icpc cl icl g++])
AC_PROG_F77([ifort g77 gfortran])
AC_PROG_FC([ifort gfortran])

#AUTOMAKE
#Initialize automake and declare foreign so that we don't need a ChangeLog, INSTALL, etc
AM_INIT_AUTOMAKE([foreign])
AC_CONFIG_HEADERS([./config.h]) #Config file must be config.h
AM_SILENT_RULES([yes]) #Do not show compilation command by default
AM_PROG_AR

#Libtool
LT_INIT

#Run issm_options.m4
ISSM_OPTIONS

#List all Makefiles
AC_CONFIG_FILES([
Makefile
src/Makefile
src/c/Makefile
src/wrappers/Makefile
src/wrappers/python/Makefile
src/wrappers/matlab/Makefile
src/wrappers/javascript/Makefile
src/mobile/Makefile
src/mobile/native/Makefile
src/m/Makefile
])

#End of configure.ac
AC_OUTPUT

The above message error seems to me that there is some definition missing in aclocal.m4.

Just to clarify, I will be working with MATLAB after I compile ISSM.

Thanks a lot for your time and help,

Ignacio

justinquinn

Hey Ignacio,
The first thing that stands out is your configuration for PETSc: you can only have one --download-fblaslapack option set, and I believe it has to be an integer bool. I have seen the error reagrding fblaslapack before and I don't think it actually has to do with downloading the package: it's tar that is malfunctioning. One way to test this is to, as you did, download the contents of the fblaslapack link manually, then try to untar it with tar -zxvf fblaslapack-3.4.2.tar.gz. It may just be that the downloaded file is corrupted somehow. I suppose a second test would be to take some file on your machine, compress it with tar, then try to decompress it. If this works, the issue is not with tar but with the file (which I think is the case). What I did in this situation was assume there was some interruption in the connection to http://ftp.mcs.anl.gov, and I just kept running the PETSc installation script until it stopped complaining (annoying, I know).

Another snag I noticed is this,
bash: ./configure.sh: Permission denied
which indicates that permissions need to be modified for your configuration file. I think you can get away with chmod 644 $ISSM_DIR/configure.sh, but you may need to adjust it to something more open, like 755.

The m4 files are fine, and I am not exactly sure what is going on with the autotools error messages. In fact, they may be misleading and it could just be that the permissions on the configure.sh file are causing the subsequent error. Are you sure that autotools installed correctly? What is the exit code at the end of cat $ISSM_DIR/extenalpackages/autotools/compil.log? Regardless, let's get PETSc installed first, correct the permissions issue, then we can see about tackling the other stuff.

iisola97

Justin,

I've changed the permissions of configure.sh to 755.
Regarding 'cat $ISSM_DIR/extenalpackages/autotools/compil.log', there is no such file in that folder.
Regarding the installation of the external packages, mpich and cmake ran fine. I did the following for petsc:
I've downloaded the tar file for fblaslapack, scalapack and metis, and untar them. After that, I changed the install-3.7-linux64.sh file to this:

#!/bin/bash
set -eu

#Some cleanup
rm -rf install petsc-3.7.6 src
mkdir install src

#Download from ISSM server
$ISSM_DIR/scripts/DownloadExternalPackage.sh 'http://issm.jpl.nasa.gov/files/externalpackages/petsc-lite-3.7.6.tar.gz' 'petsc-3.7.6.tar.gz'

#Untar and move petsc to install directory
tar -zxvf petsc-3.7.6.tar.gz
mv petsc-3.7.6/* src/
rm -rf petsc-3.7.6

#configure
cd src
./config/configure.py \
--prefix="$ISSM_DIR/externalpackages/petsc/install" \
--with-mpi-dir="$ISSM_DIR/externalpackages/mpich/install" \
--PETSC_DIR="$ISSM_DIR/externalpackages/petsc/src" \
--with-debugging=0 \
--with-valgrind=0 \
--with-x=0 \
--with-ssl=0 \
--with-shared-libraries=1 \
--download-metis="$ISSM_DIR/externalpackages/petsc/petsc-pkg-metis-0adf3ea7785d" \
--download-parmetis=1 \
--download-mumps=1 \
--download-scalapack="$ISSM_DIR/externalpackages/petsc/scalapack-2.0.2" \
--download-fblaslapack="$ISSM_DIR/externalpackages/petsc/fblaslapack-3.4.2" \
--with-pic=1

#Compile and intall
make
make install

FBLASLAPACK and SCALAPACK installations ran perfectly, however, I got this error regarding METIS:

                  ===============================================================================
                      Trying to download dir:///home/ignacio/research/mars/ISSM_source/trunk/externalpackages/petsc/petsc-pkg-metis-0adf3ea7785d for METIS
                  ===============================================================================

UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details):

Unable to download METIS
Failed to download METIS

What it seems weird to me is the following: before downloading the METIS file and adding the manual installation to the .sh file, I got this message:

                  ===============================================================================
                      Trying to download https://bitbucket.org/petsc/pkg-metis/get/v5.1.0-p4.tar.gz for METIS
                  ===============================================================================

UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details):

Unable to download METIS
Unable to download package METIS from: https://bitbucket.org/petsc/pkg-metis/get/v5.1.0-p4.tar.gz

If URL specified manually - perhaps there is a typo?
If your network is disconnected - please reconnect and rerun ./configure
Or perhaps you have a firewall blocking the download
You can run with --with-packages-dir=/adirectory and ./configure will instruct you what packages to download manually
or you can download the above URL manually, to /yourselectedlocation/v5.1.0-p4.tar.gz
and use the configure option:
--download-metis=/yourselectedlocation/v5.1.0-p4.tar.gz
*******************************************************************************

It's like waiting for a file named as: v5.1.0-p4.tar.gz. However, when I downloaded the file from https://bitbucket.org/petsc/pkg-metis/get/v5.1.0-p4.tar.gz, the file is: petsc-pkg-metis-0adf3ea7785d.tar.gz. Could this be messing up the METIS installation?

I couldn't proceed with the METIS installation and finishing installing petsc because of this.

Again, thanks for your time and help,

Ignacio

justinquinn

iisola97 Hey Ignacio,
Sorry about the bad path to the autotools compilation log: I meant $ISSM_DIR/externalpackages/autotools/src/compil.log

Yeah, there's something fishy going on here with the PETSc external packages (fblaslapack, etc). Really, you shouldn't need to download them manually: I have only ever opted for the various packages with --download-PACKAGE_NAME=1. Moreover, the output of the untarring of petsc-pkg-metis-0adf3ea7785d appears to be a JavaScript handler, which I imagine manages requests for the actual file. In other words, I do not believe you have the actual tarball there, but have the page content from that Web address. How are you downloading these packages manually? With wget or curl? Via a Web browser. In any case, I would recommend resetting all of these --download-PACKAGE_NAME options to 1, and just running the install script until it works. I know: it sounds one definition for crazy, but there is something variable here and I believe it's the actual connection to the http://ftp.mcs.anl.gov Web site. Which brings up another point: do you have OpenSSL installed? You can do so with sudo apt-get install opensslI believe. This may be the root issue for why the connections are failing.

iisola97

justinquinn Justin,

I reset all of the --download-PACKAGE_NAME options to 1 in install-3.7-linux64.sh file.
I installed OpenSSL following the steps provided on this website: https://websiteforstudents.com/manually-install-the-latest-openssl-toolkit-on-ubuntu-16-04-18-04-lts/
I didn't know I had to install autotools before mpich, cmake, petsc, etc. Autotools is not listed in $ISSM_DIR/jenkins/linux64_ross-python configuration file. However, I did. It ran perfectly. Mpich ran perfectly too, but when I tried to install cmake after, I came up with the following error:

Scanning dependencies of target cmlibarchive
[ 45%] Building C object Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_acl.c.o
[ 46%] Building C object Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_check_magic.c.o
[ 46%] Building C object Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_cmdline.c.o
[ 46%] Building C object Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_cryptor.c.o
In file included from /home/ignacio/research/mars/ISSM_source/trunk/externalpackages/cmake/install/Utilities/cmlibarchive/libarchive/archive_cryptor.c:32:0:
/home/ignacio/research/mars/ISSM_source/trunk/externalpackages/cmake/install/Utilities/cmlibarchive/libarchive/archive_cryptor_private.h:107:17: error: field ‘ctx’ has incomplete type
EVP_CIPHER_CTX ctx;
^
Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/build.make:134: recipe for target 'Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_cryptor.c.o' failed
make[2]: *** [Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_cryptor.c.o] Error 1
CMakeFiles/Makefile2:1728: recipe for target 'Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/all' failed
make[1]: *** [Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/all] Error 2
Makefile:160: recipe for target 'all' failed
make: *** [all] Error 2

I don't know if the installation of OpenSSL is causing this error because cmake installation used to ran perfectly before.

Thanks,

Ignacio

justinquinn

I didn't know I had to install autotools before mpich, cmake, petsc, etc. Autotools is not listed in $ISSM_DIR/jenkins/linux64_ross-python configuration file."

You're absolutely right, and this is curious. I'll have to look into it. Sorry about that: a better reference (especially if you are not using the Python flavor) is $ISSM_DIR/jenkins/linux64_ross.

I don't know if the installation of OpenSSL is causing this error because cmake installation used to ran perfectly before.

I am not sure about this one. To be honest, I would have just use the apt package manager to install OpenSSL as it can save you a lot of potential headaches (not package managers in general, but the globally archived packages on apt are pretty solid). I would try uninstalling the existing copy and simply running sudo apt-get install openssl.

iisola97

justinquinn
I uninstalled the existing copy of openssl and did 'sudo apt-get install openssl'
After this, I've tried installing autotools. I got this error now:

=== INSTALLING M4 ==
File m4-1.4.17.tar.gz already exists and will not be downloaded...

gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now

I've also tried installing cmake to see if the previous error was no longer showing up, but I didn't have luck, it's still showing the same error:

However, mpich installation ran perfectly.

I'm starting to think I should start from scratch again.

Thanks for your patience and help,

Ignacio

justinquinn

Hey Ignacio,
That may be a good idea: to check out a fresh copy of the repo. Then, start with autotools, using the install script listed in $ISSM_DIR/jenkins/linux64, and don’t move on to the next external package until you’re sure the previous one is installed successfully. In particular, all of the subsequent packages rely on autotools. I believe it is only PETSc that relies on CMake (but there may be others). Let me know how this works out and we’ll go from there.

iisola97

Justin,
I've started again from scratch. When I start the compilation process with:

cd $ISSM_DIR/externalpackages/autotools
./install.sh

I still get this:
=== INSTALLING M4 ==

gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now

m4-1.4.17.tar.gz file is downloaded correctly but install.sh crashes when wants to tar this file: tar -zxvf $M4_TARBALL.
The error is that the file is empty. That's the reason why 'unexpected end of file' shows up.

Do you know by chance why the downloaded file is empty?

Thanks for your help

iisola97

I've just tried downloading the .tar.gz file manually with the following link: 'https://issm.ess.uci.edu/files/externalpackages/m4-1.4.17.tar.gz' and the file downloaded isn't empty, it has all the files on it.

I did install OpenSSL using 'sudo apt-get install openssl', but for some reason it seems that this line of code within $ISSM_DIR/externalpackages/autotools/install.sh:

'$ISSM_DIR/scripts/DownloadExternalPackage.sh "https://issm.ess.uci.edu/files/externalpackages/${M4_TARBALL}" $M4_TARBALL'

is not being able to download the .tar.gz correctly and downloads it completely empty.

justinquinn

iisola97 Interesting. What do you get from running which wget?

iisola97

justinquinn I get this: /usr/bin/wget.
I found that I didn't have installed 'curl'. I've installed it but still isn't able to download any file properly.

justinquinn

Can you try cd $ISSM_DIR/externalpackages/autotools; wget -O m4-1.4.17.tar.gz https://issm.ess.uci.edu/files/externalpackages/m4-1.4.17.tar.gz; tar -zxvf m4-1.4.17.tar.gz?

iisola97

cd $ISSM_DIR/externalpackages/autotools; wget -O m4-1.4.17.tar.gz https://issm.ess.uci.edu/files/externalpackages/m4-1.4.17.tar.gz; tar -zxvf m4-1.4.17.tar.gz

--2019-06-27 13:53:40-- https://issm.ess.uci.edu/files/externalpackages/m4-1.4.17.tar.gz
Resolving issm.ess.uci.edu (issm.ess.uci.edu)... 128.200.166.142
Connecting to issm.ess.uci.edu (issm.ess.uci.edu)|128.200.166.142|:443... connected.
ERROR: cannot verify issm.ess.uci.edu's certificate, issued by ‘CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US’:
Unable to locally verify the issuer's authority.
To connect to issm.ess.uci.edu insecurely, use `--no-check-certificate'.

gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now

The file downloaded is still empty.

justinquinn

This is so weird: the installation of OpenSSL should have resolved it. Did you open a new terminal instance after installation? A restart of your machine might be necessary too.

iisola97

I restarted my machine after reinstalling OpenSSL and also after installing curl, and now I was working with a new terminal instance.

As I mentioned above, I used this URL https://websiteforstudents.com/manually-install-the-latest-openssl-toolkit-on-ubuntu-16-04-18-04-lts/ to install OpenSSL for the first time. I removed everything and then reinstalled OpenSSL using 'sudo apt-get install openssl'. I don't know if some of the settings I've changed while following the steps provided by this website are messing something up now. The thing is, after I installed OpenSSL following that website, my machine has been no longer able to download any files. I removed everything before I reinstalled OpenSSL, but it seems that something isn't fine.

justinquinn

iisola97 I think unfortunately you're going to have to download the openssl-1.1.1.tar.gz package again (as listed in that Web site), unzip it, and look through the configure and make files to see if there are any other files you may have missed when cleaning up. You may want to remove the apt-installed version first and then reinstall it afterwards to make sure there aren't any conflicts. So,

sudo apt-get remove openssl; sudo apt-get autoremove
Download the openssl-1.1.1.tar.gz package, unzip, and inspect relevant files to ensure there aren't any other artifacts (look for anything copied to /usr/local/ssl).
From that same Web site, sudo nano /etc/environment and remove the reference to /usr/local/ssl/bin.
sudo apt-get install openssl

The nuclear option (assuming this is a virtual machine) considering that you are more or less at square one in terms of the ISSM installation is to create a new VM if the above does not work. I'm sorry, I don't know what else to recommend if the OpenSSL installation is corrupted and unrecoverable.

iisola97

justinquinn I've tried your steps but didn't have luck. I'm not working with a VM so creating a new account isn't an option, unfortunately. I created a new account on this computer but I still get the same error.

Basically, when I run wget I get the following error:
ERROR: cannot verify URL's certificate, issued by ‘CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US’:
Unable to locally verify the issuer's authority.

I was doing some research and it seems that I removed some SSL certificates ('ca-certificate') somehow while uninstalling the first OpenSSL I'd installed before.

$ dpkg -l curl wget ca-certificates openssl Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-============================-===================-===================-============================================================= un ca-certificates <none> <none> (no description available) ii curl 7.47.0-1ubuntu2.13 amd64 command line tool for transferring data with URL syntax ii openssl 1.0.2g-1ubuntu4.15 amd64 Secure Sockets Layer toolkit - cryptographic utility ii wget 1.17.1-1ubuntu1.5 amd64 retrieves files from the web

This is the reason why I can't access any URL and be able to download their files (I think). I was trying to find a way to install these certificates again (I think I need one called Let's Encrypt Authority X3) but I'm a little bit lost.
I was looking at these websites in order to solve this issue:

I don't know if you can help me with this. Nonetheless, thanks a lot for your patience and time throughout these days.

justinquinn

iisola97 Hi Ignacio,
No worries: I'm happy to help and am sorry this process has been so frustrating for you.

You shouldn't need Let's Encrypt: that package is more for creating SSL certificates on the server side.

Can you try this: sudo update-ca-certificates?