Building the CHACO and DAKOTA packages

iisola97

Where can I find the DAKOTA and CHACO external packages?

mathieumorlighem

Hi Ignacio,
again, we need more info. Are you using the binaries or are you compiling the code from scratch?

if you are compiling the code yourself (i.e. you checked out the code from svn), all the external packages are in $ISSM_DIR/externalpackages, you will find Dakota and Chaco there
if you are using the binaries, make sure to download the ones that support dakota (macOS for now)

Thanks
Mathieu

iisola97

Hi Mathieu,
Thanks for your reply.
I was using the binaries and I think external packages are not available for Linux.

justinquinn

iisola97
Hi Ignacio,
It may of interest to you that we have added Linux with Dakota binaries, which are available here: https://issm.jpl.nasa.gov/download/binaries/. Please let me know if you do end up downloading them and run into any issues.

mathieumorlighem

Hi Ignacio,
you are right, you cannot compile external packages if you are using binaries, and Dakota is not yet available for linux binaries. The only way is to download the source code and compile everything yourself. Alternatively, we can ask Justin to look into compiling the binaries statically for linux but I don't know how long this would take.
Best,
Mathieu

justinquinn

mathieumorlighem If Linux + Dakota binaries are a priority, I am happy to work on it. I have already sorted out a similar configuration on MacOS, and we do have a Linux + Dakota build, so it should not be too bad.

iisola97

justinquinn Thanks! I don't think I will be using Dakota (I just wanted to complete all the tutorials), but if I do, I 'll let you know here in the forum.

kimyoungjin06

Hello!
I use ubuntu 18.04.

In installing DAKOTA(=6.2-linux64.sh),
*** I install boost=1.55-linux with 58 failed and 12 skipped(?).
*** so, I install boost=1.55-linux64-static with 1 failed and 12(?) skipped.
*** what is correct choice in my case?
I get error message "cannot find -lmpi" among cmake(?)

So, I choose another version 5.1,
But in that case, it need to install lapack.
I get this message

CMake Error at CMakeLists.txt:170 (find_package):
  Could not find a package configuration file provided by "LAPACK" with any
  of the following names:

    LAPACKConfig.cmake
    lapack-config.cmake

  Add the installation prefix of "LAPACK" to CMAKE_PREFIX_PATH or set
  "LAPACK_DIR" to a directory containing one of the above files.  If "LAPACK"
  provides a separate development package or SDK, be sure it has been
  installed.

How can I finish to install dakota?

And,
in this case, there is needed for the configuration option?
like this "--with-lapack-dir=$ISSM_DIR/externalpackages/lapack/install \"

Last question is I already install without dakota, chaco, (and boost) with following tutorial installing documentation.

justinquinn

kimyoungjin06
Hey there,
In the first case, the CMake cannot find your 'mpi' library (probably actually called libmpi.a or libmpi.so based on it being a Linux system). A few possible reasons for this are,

the CMake configuration files (CMakeLists.txt) for the project are not set up to look in the correct location.
you have not supplied a path to you MPI library directory with the corresponding option (something like "CMAKE_MPI_LIBRARIES"; again, see the CMakeLists.txt files).
you really do not have an MPI implementation installed.

If you do not have an MPI implementation installed, you can do so by installing MPICH after Autotools and CMake, but before any of the other packages.

In the case of Dakota v5.1, you are correct about needing the configuration option, but I believe here you are referring to configuration of the ISSM installation and not the Dakota external package. What you will want to do (assuming the above is correct) is,

install PETSc before Dakota, making sure that the --download-fblaslapack=1 option is supplied.
modify your installation script or command line options for Dakota to ensure that it has the path to LAPACK, which will be something like --with-lapack-dir=$ISSM_DIR/externalpackages/petsc/install.

Let me know if you run into any other issues!

kimyoungjin06

justinquinn
Thanks for your rapid response!

last night, I'm so tired, so there is not sufficient description for questions.

Q1 libmpi is not awared

This is a computer with newly installing ISSM.

Ubuntu 18.04
I follow these script

apt-get update
apt-get install -y subversion gcc g++ gfortran make perl python patch
svn --username anon --password anon checkout http://issm.ess.uci.edu/svn/issm/issm/trunk

cd /home/adminn/issm
export ISSM_DIR=/home/adminn/issm/trunk
cd $ISSM_DIR

# Need gcc g++ gfortran make
cd $ISSM_DIR/externalpackages/mpich
./install-3.2-linux64.sh
source $ISSM_DIR/etc/environment.sh 

# Need perl
cd $ISSM_DIR/externalpackages/autotools
./install.sh

cd $ISSM_DIR/externalpackages/metis
./install-4.0-linux64.sh

cd $ISSM_DIR/externalpackages/cmake
./install.sh

# Need metis log message without installing metis
'''
From ../externalpackages/petsc/src/.configure.log
================================================================================
TEST configureDirectories from PETSc.options.petscdir(/home/adminn/issm/trunk/externalpackages/petsc/src/config/PETSc/options/petscdir.py:23)
TESTING: configureDirectories from PETSc.options.petscdir(config/PETSc/options/petscdir.py:23)
  Checks PETSC_DIR and sets if not set
*******************************************************************************
         UNABLE to CONFIGURE with GIVEN OPTIONS    (see configure.log for details):
-------------------------------------------------------------------------------
The environmental variable PETSC_DIR /home/adminn/issm/trunk/externalpackages/petsc/install MUST be the current directory /home/adminn/issm/trunk/externalpackages/petsc/src
*******************************************************************************
'''
cd $ISSM_DIR/externalpackages/petsc
./install-3.9-linux64.sh

# Need python python3(?)
cd $ISSM_DIR/externalpackages/m1qn3
./install.sh

# Need patch 
cd $ISSM_DIR/externalpackages/triangle
./install-linux64.sh

source $ISSM_DIR/etc/environment.sh

PATH=$PATH:/home/issm/trunk/externalpackages/autotools/install/bin/
cd $ISSM_DIR
autoreconf -ivf

## MAKE configure.sh
'''
./configure \
--prefix=$ISSM_DIR \
--with-matlab-dir="/usr/local/MATLAB/R2019a/" \
--with-triangle-dir="$ISSM_DIR/externalpackages/triangle/install" \
--with-mpi-include="$ISSM_DIR/externalpackages/mpich/install/include" \
--with-mpi-libflags="-L$ISSM_DIR/externalpackages/mpich/install/lib/ -lmpi -lmpicxx -lmpifort" \
--with-petsc-dir="$ISSM_DIR/externalpackages/petsc/install" \
--with-metis-dir="$ISSM_DIR/externalpackages/petsc/install" \
--with-blas-lapack-dir="$ISSM_DIR/externalpackages/petsc/install" \
--with-scalapack-dir="$ISSM_DIR/externalpackages/petsc/install/" \
--with-mumps-dir="$ISSM_DIR/externalpackages/petsc/install/" \
--with-m1qn3-dir="$ISSM_DIR/externalpackages/m1qn3/install" \
--with-numthreads=2
'''

chmod 755 configure.sh

./configure.sh

make

Before 'make', there is no any error.
log message

CXXLD    libISSMCore.la

*** Warning: Linking the shared library libISSMCore.la against the
*** static library /home/testuser/issm/trunk/externalpackages/m1qn3/install/libm1qn3.a is not portable!

*** Warning: Linking the shared library libISSMCore.la against the
*** static library /home/testuser/issm/trunk/externalpackages/m1qn3/install/libddot.a is not portable!
libtool: warning: library '/home/testuser/issm/trunk/externalpackages/mpich/install/lib//libmpi.la' was moved.
libtool: warning: library '/home/testuser/issm/trunk/externalpackages/mpich/install/lib//libmpicxx.la' was moved.
libtool: warning: library '/home/testuser/issm/trunk/externalpackages/mpich/install/lib//libmpifort.la' was moved.
ar: `u' modifier ignored since `D' is the default (see `U')
/bin/sed: can't read /home/issm/trunk/externalpackages/mpich/install/lib/libmpi.la: No such file or directory
libtool:   error: '/home/issm/trunk/externalpackages/mpich/install/lib/libmpi.la' is not a valid libtool archive
Makefile:4009: recipe for target 'libISSMCore.la' failed
make[3]: *** [libISSMCore.la] Error 1
make[3]: Leaving directory '/home/testuser/issm/trunk/src/c'
Makefile:462: recipe for target 'all-recursive' failed
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory '/home/testuser/issm/trunk/src'
Makefile:583: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/home/testuser/issm/trunk'
Makefile:480: recipe for target 'all' failed
make: *** [all] Error 2

But in the target directory, there exist these files you said.

user01@opendataserver:/home/testuser/issm/trunk$ find -name "libmpi*"
find: ‘./externalpackages/metis/install/Doc’: Permission denied
find: ‘./externalpackages/metis/install/Graphs’: Permission denied
find: ‘./externalpackages/metis/install/Lib’: Permission denied
find: ‘./externalpackages/metis/install/Programs’: Permission denied
find: ‘./externalpackages/metis/install/Test’: Permission denied
./externalpackages/mpich/src/lib/.libs/libmpi.so.12.1.0
./externalpackages/mpich/src/lib/.libs/libmpi.so.12
./externalpackages/mpich/src/lib/.libs/libmpi.so
./externalpackages/mpich/src/lib/.libs/libmpi.a
./externalpackages/mpich/src/lib/.libs/libmpi.lai
./externalpackages/mpich/src/lib/.libs/libmpi.la
./externalpackages/mpich/src/lib/.libs/libmpifort.so.12
./externalpackages/mpich/src/lib/.libs/libmpifort.so
./externalpackages/mpich/src/lib/.libs/libmpifort.a
./externalpackages/mpich/src/lib/.libs/libmpifort.lai
./externalpackages/mpich/src/lib/.libs/libmpifort.la
./externalpackages/mpich/src/lib/.libs/libmpicxx.so.12
./externalpackages/mpich/src/lib/.libs/libmpicxx.so
./externalpackages/mpich/src/lib/.libs/libmpicxx.a
./externalpackages/mpich/src/lib/.libs/libmpicxx.lai
./externalpackages/mpich/src/lib/.libs/libmpicxx.la
./externalpackages/mpich/src/lib/.libs/libmpifort.so.12.1.0T
./externalpackages/mpich/src/lib/.libs/libmpifort.so.12.1.0
./externalpackages/mpich/src/lib/.libs/libmpicxx.so.12.1.0T
./externalpackages/mpich/src/lib/.libs/libmpicxx.so.12.1.0
./externalpackages/mpich/src/lib/libmpi.la
./externalpackages/mpich/src/lib/libmpifort.la
./externalpackages/mpich/src/lib/libmpicxx.la
./externalpackages/mpich/install/lib/libmpi.so.12.1.0
./externalpackages/mpich/install/lib/libmpi.so.12
./externalpackages/mpich/install/lib/libmpi.so
./externalpackages/mpich/install/lib/libmpi.la
./externalpackages/mpich/install/lib/libmpifort.so.12.1.0
./externalpackages/mpich/install/lib/libmpifort.so.12
./externalpackages/mpich/install/lib/libmpifort.so
./externalpackages/mpich/install/lib/libmpifort.la
./externalpackages/mpich/install/lib/libmpicxx.so.12.1.0
./externalpackages/mpich/install/lib/libmpicxx.so.12
./externalpackages/mpich/install/lib/libmpicxx.so
./externalpackages/mpich/install/lib/libmpicxx.la
./externalpackages/mpich/install/lib/libmpi.a
./externalpackages/mpich/install/lib/libmpifort.a
./externalpackages/mpich/install/lib/libmpicxx.a
./externalpackages/mpich/install/lib/libmpich.so
./externalpackages/mpich/install/lib/libmpichf90.so
./externalpackages/mpich/install/lib/libmpichcxx.so
``` 

- and libtool is installed

```bash
user01@opendataserver:/home/testuser/issm/trunk$ libtool --version
libtool (GNU libtool) 2.4.2
Written by Gordon Matzigkeit <gord@gnu.ai.mit.edu>, 1996

Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Q2 add Chaco and DAKOTA in a computer already finish to install the ISSM

Environment and situation

Ubuntu 18.04
Already finish installing ISSM without Chaco and DAKOTA
Want to add Chaco and DAKOTA

Original question I want to

In this case, there is no uninstall ISSM or re-install external packages?
Why I get error message cannot find -lmpi among cmake in the installing DAKOTA.
Is there any difference or critical disadvantage for 5.1 than 6.2 DAKOTA.

My understand

install PETSc before Dakota, making sure that the --download-fblaslapack=1 option is supplied.
- Is that mean newly install PETSc before Dakota with the following command?
- ./install-3.9-linux64.sh --download-fblaslapack=1
- And in this case, there does not need to install /externalpackages/lapack?

justinquinn

kimyoungjin06
Hi, there is a lot here, so I will try to get to everything.

Q1

That is indeed very strange. In particular, I have never seen a message like this one,

libtool: warning: library '/home/testuser/issm/trunk/externalpackages/mpich/install/lib//libmpi.la' was moved.

specifically, the part where it says the library was moved. The only advice I could give in this case is to try to,
rm -rf $ISSM_DIR/externalpackages/mpich/install and try reinstalling. But it is probably better to just skip to Q2.

Q2

There should be no need to reinstall the other external packages. From our nightly build that has Dakota, what you will need the following external packages (installed in this order),

autotools install.sh
cmake install.sh
mpich install-3.2-linux64.sh
petsc install-3.7-linux64.sh
triangle install-linux64.sh
boost install-1.55-linux.sh
dakota install-6.2-linux64.sh
chaco install.sh
m1qn3 install.sh
shell2junit install.sh

NOTE: I think you are already aware of this, but be sure to source $ISSM_DIR/etc/environment.sh after installing each package.

Then, run configure with the following options,

--prefix=$ISSM_DIR \
--disable-static \
--with-matlab-dir="/usr/local/MATLAB/R2019a" \
--with-mpi-include="$ISSM_DIR/externalpackages/mpich/install/include" \
--with-mpi-libflags="-L$ISSM_DIR/externalpackages/mpich/install/lib -lmpi -lmpicxx -lmpifort" \
--with-fortran-lib="-L/usr/lib/gcc/x86_64-linux-gnu/4.9 -lgfortran" \
--with-cxxoptflags="-std=c++11" \
--with-petsc-dir="$ISSM_DIR/externalpackages/petsc/install" \
--with-metis-dir="$ISSM_DIR/externalpackages/petsc/install" \
--with-blas-lapack-dir="$ISSM_DIR/externalpackages/petsc/install" \
--with-scalapack-dir="$ISSM_DIR/externalpackages/petsc/install" \
--with-mumps-dir="$ISSM_DIR/externalpackages/petsc/install" \
--with-triangle-dir="$ISSM_DIR/externalpackages/triangle/install" \
--with-boost-dir="$ISSM_DIR/externalpackages/boost/install" \
--with-dakota-dir="$ISSM_DIR/externalpackages/dakota/install" \
--with-chaco-dir="$ISSM_DIR/externalpackages/chaco/install" \
--with-m1qn3-dir="$ISSM_DIR/externalpackages/m1qn3/install" \
--with-numthreads=2

NOTE: You may need to modify the path to libgfortran.

That should work, but let me knw if you run into any other issues.

kimyoungjin06

mathieumorlighem Thanks!
I have a specific error with installing CMake. But in that case, I can install within a docker container with volume sharing.
So, I can finish all the processes to install.
And root permission is for installation with docker container ( for using docker, it is needed a root.)

And also appreciate to justinquinn !!

kimyoungjin06

Additional Question about root and sudo permission

my account is sudoer, but I do not use root

But in this case, export ISSM_DIR... is not awared

  user01@opendataserver:/home/testuser/issm/trunk/externalpackages/autotools$ sudo ./install.sh 
[sudo] password for user01: 
./install.sh: line 25: ISSM_DIR: unbound variable
user01@opendataserver:/home/testuser/issm/trunk/externalpackages/autotools$ $ISSM_DIR/
-bash: /home/testuser/issm/trunk/: Is a directory

Additional Question about docker image

I want to make a docker image for an installation ISSM. But I do not have a sufficient understanding of installation.
And in my personal ubuntu machine, fortunately, I succeed to install ISSM with docker with the following description.

All script is done in docker container before configuring ~ ./configure.sh
Volume sharing $ISSM_DIR ~ .../issm/trunk in/out docker container
For linking with matlab, configure.sh and make is excuted out of docker container.

Q2.1

Is it possible installation with docker container with my method?
I think that many points are wondering for in/out the difference in the environment.

Q2.2

I consider if additional volume sharing <matlab path>. And all processes including configure.sh and make are excuted in the docker container. Finally, I want to run matlab out of the docker container. Is it possible?

mathieumorlighem

Hi, if you cannot set the variable ISSM_DIR, nothing will ever work and that may explain why you have these error messages. You do NOT need to be root to be able to set variables (you can try to type export ISSM_DIR=".....trunk" in the terminal and it will work. That being said, I have never used Docker so I don't know whether installing ISSM is even possible, especially with heavy packages like Dakota...