Compiling ISSM on Ubuntu 16.04.5

iisola97

justinquinn
I uninstalled the existing copy of openssl and did 'sudo apt-get install openssl'
After this, I've tried installing autotools. I got this error now:

=== INSTALLING M4 ==
File m4-1.4.17.tar.gz already exists and will not be downloaded...

gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now

I've also tried installing cmake to see if the previous error was no longer showing up, but I didn't have luck, it's still showing the same error:

Scanning dependencies of target cmlibarchive
[ 45%] Building C object Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_acl.c.o
[ 46%] Building C object Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_check_magic.c.o
[ 46%] Building C object Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_cmdline.c.o
[ 46%] Building C object Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_cryptor.c.o
In file included from /home/ignacio/research/mars/ISSM_source/trunk/externalpackages/cmake/install/Utilities/cmlibarchive/libarchive/archive_cryptor.c:32:0:
/home/ignacio/research/mars/ISSM_source/trunk/externalpackages/cmake/install/Utilities/cmlibarchive/libarchive/archive_cryptor_private.h:107:17: error: field ‘ctx’ has incomplete type
EVP_CIPHER_CTX ctx;
^
Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/build.make:134: recipe for target 'Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_cryptor.c.o' failed
make[2]: *** [Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_cryptor.c.o] Error 1
CMakeFiles/Makefile2:1728: recipe for target 'Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/all' failed
make[1]: *** [Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/all] Error 2
Makefile:160: recipe for target 'all' failed
make: *** [all] Error 2

However, mpich installation ran perfectly.

I'm starting to think I should start from scratch again.

Thanks for your patience and help,

Ignacio

justinquinn

Hey Ignacio,
That may be a good idea: to check out a fresh copy of the repo. Then, start with autotools, using the install script listed in $ISSM_DIR/jenkins/linux64, and don’t move on to the next external package until you’re sure the previous one is installed successfully. In particular, all of the subsequent packages rely on autotools. I believe it is only PETSc that relies on CMake (but there may be others). Let me know how this works out and we’ll go from there.

iisola97

Justin,
I've started again from scratch. When I start the compilation process with:

cd $ISSM_DIR/externalpackages/autotools
./install.sh

I still get this:
=== INSTALLING M4 ==

gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now

m4-1.4.17.tar.gz file is downloaded correctly but install.sh crashes when wants to tar this file: tar -zxvf $M4_TARBALL.
The error is that the file is empty. That's the reason why 'unexpected end of file' shows up.

Do you know by chance why the downloaded file is empty?

Thanks for your help

iisola97

I've just tried downloading the .tar.gz file manually with the following link: 'https://issm.ess.uci.edu/files/externalpackages/m4-1.4.17.tar.gz' and the file downloaded isn't empty, it has all the files on it.

I did install OpenSSL using 'sudo apt-get install openssl', but for some reason it seems that this line of code within $ISSM_DIR/externalpackages/autotools/install.sh:

'$ISSM_DIR/scripts/DownloadExternalPackage.sh "https://issm.ess.uci.edu/files/externalpackages/${M4_TARBALL}" $M4_TARBALL'

is not being able to download the .tar.gz correctly and downloads it completely empty.

justinquinn

iisola97 Interesting. What do you get from running which wget?

iisola97

justinquinn I get this: /usr/bin/wget.
I found that I didn't have installed 'curl'. I've installed it but still isn't able to download any file properly.

justinquinn

Can you try cd $ISSM_DIR/externalpackages/autotools; wget -O m4-1.4.17.tar.gz https://issm.ess.uci.edu/files/externalpackages/m4-1.4.17.tar.gz; tar -zxvf m4-1.4.17.tar.gz?

iisola97

cd $ISSM_DIR/externalpackages/autotools; wget -O m4-1.4.17.tar.gz https://issm.ess.uci.edu/files/externalpackages/m4-1.4.17.tar.gz; tar -zxvf m4-1.4.17.tar.gz

--2019-06-27 13:53:40-- https://issm.ess.uci.edu/files/externalpackages/m4-1.4.17.tar.gz
Resolving issm.ess.uci.edu (issm.ess.uci.edu)... 128.200.166.142
Connecting to issm.ess.uci.edu (issm.ess.uci.edu)|128.200.166.142|:443... connected.
ERROR: cannot verify issm.ess.uci.edu's certificate, issued by ‘CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US’:
Unable to locally verify the issuer's authority.
To connect to issm.ess.uci.edu insecurely, use `--no-check-certificate'.

gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now

The file downloaded is still empty.

justinquinn

This is so weird: the installation of OpenSSL should have resolved it. Did you open a new terminal instance after installation? A restart of your machine might be necessary too.

iisola97

I restarted my machine after reinstalling OpenSSL and also after installing curl, and now I was working with a new terminal instance.

As I mentioned above, I used this URL https://websiteforstudents.com/manually-install-the-latest-openssl-toolkit-on-ubuntu-16-04-18-04-lts/ to install OpenSSL for the first time. I removed everything and then reinstalled OpenSSL using 'sudo apt-get install openssl'. I don't know if some of the settings I've changed while following the steps provided by this website are messing something up now. The thing is, after I installed OpenSSL following that website, my machine has been no longer able to download any files. I removed everything before I reinstalled OpenSSL, but it seems that something isn't fine.

justinquinn

iisola97 I think unfortunately you're going to have to download the openssl-1.1.1.tar.gz package again (as listed in that Web site), unzip it, and look through the configure and make files to see if there are any other files you may have missed when cleaning up. You may want to remove the apt-installed version first and then reinstall it afterwards to make sure there aren't any conflicts. So,

sudo apt-get remove openssl; sudo apt-get autoremove
Download the openssl-1.1.1.tar.gz package, unzip, and inspect relevant files to ensure there aren't any other artifacts (look for anything copied to /usr/local/ssl).
From that same Web site, sudo nano /etc/environment and remove the reference to /usr/local/ssl/bin.
sudo apt-get install openssl

The nuclear option (assuming this is a virtual machine) considering that you are more or less at square one in terms of the ISSM installation is to create a new VM if the above does not work. I'm sorry, I don't know what else to recommend if the OpenSSL installation is corrupted and unrecoverable.

iisola97

justinquinn I've tried your steps but didn't have luck. I'm not working with a VM so creating a new account isn't an option, unfortunately. I created a new account on this computer but I still get the same error.

Basically, when I run wget I get the following error:
ERROR: cannot verify URL's certificate, issued by ‘CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US’:
Unable to locally verify the issuer's authority.

I was doing some research and it seems that I removed some SSL certificates ('ca-certificate') somehow while uninstalling the first OpenSSL I'd installed before.

$ dpkg -l curl wget ca-certificates openssl Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-============================-===================-===================-============================================================= un ca-certificates <none> <none> (no description available) ii curl 7.47.0-1ubuntu2.13 amd64 command line tool for transferring data with URL syntax ii openssl 1.0.2g-1ubuntu4.15 amd64 Secure Sockets Layer toolkit - cryptographic utility ii wget 1.17.1-1ubuntu1.5 amd64 retrieves files from the web

This is the reason why I can't access any URL and be able to download their files (I think). I was trying to find a way to install these certificates again (I think I need one called Let's Encrypt Authority X3) but I'm a little bit lost.
I was looking at these websites in order to solve this issue:

I don't know if you can help me with this. Nonetheless, thanks a lot for your patience and time throughout these days.

justinquinn

iisola97 Hi Ignacio,
No worries: I'm happy to help and am sorry this process has been so frustrating for you.

You shouldn't need Let's Encrypt: that package is more for creating SSL certificates on the server side.

Can you try this: sudo update-ca-certificates?

iisola97

justinquinn I really appreciate your help. Thanks, Justin.
This is what I got:

$ sudo update-ca-certificates [sudo] password for ignacio: sudo: update-ca-certificates: command not found

justinquinn

iisola97 Okay, another thing occurred to me: I'm guessing that when you uninstalled the manually-installed version of OpenSSL, you removed /usr/local/ssl entirely? Regardless, let's try restoring missing files if possible with sudo apt-get update; sudo apt-get upgrade, then test wget/curl again. Let me know what happens.

iisola97

justinquinn There are still some files under /usr/local/ssl:
/usr/local/ssl$ ls -l total 44 drwxr-xr-x 2 root root 4096 Jun 26 11:03 certs -rw-r--r-- 1 root root 414 Jun 26 11:03 ct_log_list.cnf -rw-r--r-- 1 root root 414 Jun 26 11:03 ct_log_list.cnf.dist drwxr-xr-x 2 root root 4096 Jun 26 11:03 misc -rw-r--r-- 1 root root 10998 Jun 26 11:03 openssl.cnf -rw-r--r-- 1 root root 10998 Jun 26 11:03 openssl.cnf.dist drwxr-xr-x 2 root root 4096 Jun 26 11:03 private

I ran sudo apt-get update; sudo apt-get upgrade, and got a long list of commands being executed and at the end this:

Fetched 186 MB in 10s (17.2 MB/s) E: Failed to fetchhttps://repo.skype.com/deb/pool/main/s/skypeforlinux/skypeforlinux_8.47.0.59_amd64.deb`

E: Unable to fetch some archives, try running apt-get update or apt-get --fix-missing. ignacio@mangala:~$ wget https://www.openssl.org/source/openssl-1.1.1.tar.gz --2019-06-27 19:16:26-- https://www.openssl.org/source/openssl-1.1.1.tar.gz Resolving www.openssl.org (www.openssl.org)... 104.93.192.141, 2600:140a:0:399::c1e, 2600:140a:0:389::c1e Connecting to www.openssl.org (www.openssl.org)|104.93.192.141|:443... connected. ERROR: cannot verify www.openssl.org's certificate, issued by ‘CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US’: Unable to locally verify the issuer's authority. To connect to www.openssl.org insecurely, use--no-check-certificate'.`

justinquinn

iisola97 Okay, how about sudo apt-get install ca-certificates?

iisola97

justinquinn it worked!!!
Autotools installed perfectly. I will move onto the next external packages and let you know if I come up with any error.
Thanks a lot!!

justinquinn

Nice!!! Making progress 🙂 Yep, keep me posted.

iisola97

I am following the steps provided in $ISSM_DIR/jenkins/linux64_ross.
As I said, autotools ran perfectly. The next external package is cmake, and I got this:

Scanning dependencies of target cmlibarchive [ 45%] Building C object Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_acl.c.o [ 46%] Building C object Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_check_magic.c.o [ 46%] Building C object Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_cmdline.c.o [ 46%] Building C object Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_cryptor.c.o In file included from /home/ignacio/research/mars/ISSM_source/trunk/externalpackages/cmake/install/Utilities/cmlibarchive/libarchive/archive_cryptor.c:32:0: /home/ignacio/research/mars/ISSM_source/trunk/externalpackages/cmake/install/Utilities/cmlibarchive/libarchive/archive_cryptor_private.h:107:17: error: field ‘ctx’ has incomplete type EVP_CIPHER_CTX ctx; ^ Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/build.make:134: recipe for target 'Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_cryptor.c.o' failed make[2]: *** [Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/archive_cryptor.c.o] Error 1 CMakeFiles/Makefile2:1728: recipe for target 'Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/all' failed make[1]: *** [Utilities/cmlibarchive/libarchive/CMakeFiles/cmlibarchive.dir/all] Error 2 Makefile:160: recipe for target 'all' failed make: *** [all] Error 2

justinquinn

iisola97 From what I can see, this is an issue with the latest currently available version of OpenSSL on apt. Let's try this: sudo apt-get remove openssl; sudo apt-get install openssl=1.0.2. If the installation fails, check apt-cache madison openssl to see what versions are available that are closest to 1.0.2, modify the second command and rerun it.