Problems when installing ISSM

wudipao88

Hello all,

I have previously run simulations based on the ISSM binaries and they performed well. However, the binaries has significant limitations, such as it does not customize the basal shear stress τ_b in the basal friction by itself, and there is no pressure melt term in the SHAKTI hydrology model. Therefore, I decided to try to install ISSM from source code in Windows. But now I have problems installing external packages. When I installed "install-3.14-win-msys2-mingw-msmpi.sh" in the petsc folder, the following error message appeared:

install-3.14-win-msys2-mingw-msmpi.sh: line 62: MPIEXEC_DIR: unbound variable

When I installed "install-win-msys2-mingw.sh" in the triangle folder, the following error message appeared:

patching file src/triangle.c
Hunk #1 FAILED at 340 (different line endings).
Hunk #2 FAILED at 938 (different line endings).
Hunk #3 FAILED at 1111 (different line endings).
Hunk #4 FAILED at 1158 (different line endings).
Hunk #5 FAILED at 3891 (different line endings).
Hunk #6 FAILED at 3899 (different line endings).
Hunk #7 FAILED at 4008 (different line endings).
Hunk #8 FAILED at 4033 (different line endings).
Hunk #9 FAILED at 4092 (different line endings).
Hunk #10 FAILED at 4129 (different line endings).
Hunk #11 FAILED at 4141 (different line endings).
Hunk #12 FAILED at 4197 (different line endings).
Hunk #13 FAILED at 4226 (different line endings).
Hunk #14 FAILED at 4586 (different line endings).
Hunk #15 FAILED at 4603 (different line endings).
Hunk #16 FAILED at 7649 (different line endings).
Hunk #17 FAILED at 7721 (different line endings).
17 out of 17 hunks FAILED -- saving rejects to file src/triangle.c.rej

When I installed "install.sh" in the m1qn3 folder, the following error message appeared:

bash: install.sh: No such file or directory

How should the above problem be solved?

Thanks !

justinquinn

wudipao88

Sorry to hear you're running into issues here. Firstly, I assume you're following the build instructions on https://issm.jpl.nasa.gov/download/windows/.

Did you run source $ISSM_DIR/etc/environment.sh before running the PETSc install script?

I'll check on the Triangle patch. It's working on our regression testing node, but we may be missing the error/warning there. Regardless, the tests pass, so it's probably not critical.

Regarding, M1QN3, yes, that should be install-win-msys2-mingw.sh. I will update the documentation.

justinquinn

I'm not sure what the issue with Triangle is. Perhaps the wrong install script was mistakenly used? This is the output I'm getting,

# ./install-win-msys2-mingw.sh
Archive:  triangle.zip
  inflating: src/A.poly
  inflating: src/README
  inflating: src/makefile
  inflating: src/showme.c
  inflating: src/triangle.c
  inflating: src/triangle.h
  inflating: src/tricall.c
patching file src/triangle.c
patching file src/triangle.h
patching file src/triangle.h
Hunk #1 succeeded at 253 (offset -6 lines).
gcc  -DNO_TIMER -fpic -DTRILIBRARY -shared -o libtriangle.dll triangle.c

Also, just to sure, you're using the 'MSYS2 MinGW 64-bit' app and running as Administrator?

wudipao88

justinquinn

Thank you for your quick reply ! Yes, I run MSYS2 MINGW64 as administrator. I also followed the instructions in the document (https://issm.jpl.nasa.gov/download/windows/) for installation. Everything went well up until the "External package installation" step, but from there, problems began. I executed "$source $ISSM_DIR/etc/environment.sh" before installing these external packages. Now, I tried the installation of these external packages again. The path to my ISSM is "E:/ISSM".

When I installed "petsc", the following message appeared after a large list of files was displayed:

===============================================================================

Configuring PETSc to compile on your system

TypeError or ValueError possibly related to ERROR in COMMAND LINE ARGUMENT while running ./configure

Invalid directory: E for key with-mpi-include

File "/e/ISSM/externalpackages/petsc/src/./config/configure.py", line 453, in petsc_configure
framework = config.framework.Framework(['--configModules=PETSc.Configure','--optionsModule=config.compilerOptions']+sys.argv[1:], loadArgDB = 0)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/framework.py", line 107, in init
self.createChildren()
File "/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/framework.py", line 324, in createChildren
self.getChild(moduleName)
File "/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/framework.py", line 309, in getChild
config.setupDependencies(self)
File "/e/ISSM/externalpackages/petsc/src/config/PETSc/Configure.py", line 80, in setupDependencies
self.blasLapack = framework.require('config.packages.BlasLapack',self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/framework.py", line 329, in require
config = self.getChild(moduleName, keywordArgs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/framework.py", line 309, in getChild
config.setupDependencies(self)
File "/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/packages/BlasLapack.py", line 21, in setupDependencies
config.package.Package.setupDependencies(self, framework)
File "/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/package.py", line 149, in setupDependencies
self.mpi = framework.require('config.packages.MPI',self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/framework.py", line 329, in require
config = self.getChild(moduleName, keywordArgs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/framework.py", line 307, in getChild
config.setup()
File "/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/base.py", line 94, in setup
script.Script.setup(self)
File "/e/ISSM/externalpackages/petsc/src/config/BuildSystem/script.py", line 108, in setup
logger.Logger.setup(self)
File "/e/ISSM/externalpackages/petsc/src/config/BuildSystem/logger.py", line 75, in setup
args.ArgumentProcessor.setup(self)
File "/e/ISSM/externalpackages/petsc/src/config/BuildSystem/args.py", line 76, in setup
self.setupArguments(self.argDB)
File "/e/ISSM/externalpackages/petsc/src/config/BuildSystem/script.py", line 92, in setupArguments
self.setupHelp(self.help)
File "/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/packages/MPI.py", line 63, in setupHelp
config.package.Package.setupHelp(self,help)
File "/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/package.py", line 159, in setupHelp
help.addArgument(self.PACKAGE,'-with-'+self.package+'-include=<dirs>',nargs.ArgDirList(None,None,'Indicate the directory of the '+self.name+' include files'))
File "/e/ISSM/externalpackages/petsc/src/config/BuildSystem/help.py", line 108, in addArgument
self.argDB.setType(self.getArgName(name), argType, forceLocal = 1)
File "/e/ISSM/externalpackages/petsc/src/config/BuildSystem/RDict.py", line 222, in setType
value.setValue(v.getValue())
File "/e/ISSM/externalpackages/petsc/src/config/BuildSystem/nargs.py", line 314, in setValue
raise ValueError('Invalid directory: '+str(dir)+' for key '+str(self.key))
makefile:15: arch-mswin-c-opt/lib/petsc/conf/petscvariables: No such file or directory
make[1]: *** No rule to make target 'arch-mswin-c-opt/lib/petsc/conf/petscvariables'. Stop.
gmakefile:5: arch-mswin-c-opt/lib/petsc/conf/petscvariables: No such file or directory
make: *** [GNUmakefile:17: arch-mswin-c-opt/lib/petsc/conf/petscvariables] Error 2
makefile:15: arch-mswin-c-opt/lib/petsc/conf/petscvariables: No such file or directory
make[1]: *** No rule to make target 'arch-mswin-c-opt/lib/petsc/conf/petscvariables'. Stop.
gmakefile:5: arch-mswin-c-opt/lib/petsc/conf/petscvariables: No such file or directory
make: *** [GNUmakefile:17: arch-mswin-c-opt/lib/petsc/conf/petscvariables] Error 2

When I installed "triangle", the following options appeared. I selected "A" and received the following prompt:

./install-win-msys2-mingw.sh

File triangle.zip already exists and will not be downloaded...
Archive: triangle.zip
replace src/A.poly? [y]es, [n]o, [A]ll, [N]one, [r]ename: A

inflating: src/A.poly
inflating: src/README
inflating: src/makefile
inflating: src/showme.c
inflating: src/triangle.c
inflating: src/triangle.h
inflating: src/tricall.c
patching file src/triangle.c
Hunk #1 FAILED at 340 (different line endings).
Hunk #2 FAILED at 938 (different line endings).
Hunk #3 FAILED at 1111 (different line endings).
Hunk #4 FAILED at 1158 (different line endings).
Hunk #5 FAILED at 3891 (different line endings).
Hunk #6 FAILED at 3899 (different line endings).
Hunk #7 FAILED at 4008 (different line endings).
Hunk #8 FAILED at 4033 (different line endings).
Hunk #9 FAILED at 4092 (different line endings).
Hunk #10 FAILED at 4129 (different line endings).
Hunk #11 FAILED at 4141 (different line endings).
Hunk #12 FAILED at 4197 (different line endings).
Hunk #13 FAILED at 4226 (different line endings).
Hunk #14 FAILED at 4586 (different line endings).
Hunk #15 FAILED at 4603 (different line endings).
Hunk #16 FAILED at 7649 (different line endings).
Hunk #17 FAILED at 7721 (different line endings).
17 out of 17 hunks FAILED -- saving rejects to file src/triangle.c.rej

When I install "m1qn3", the prompt is as follows:

./install-win-msys2-mingw.sh

File m1qn3-3.3-distrib.tgz already exists and will not be downloaded...
(Stripping trailing CRs from patch; use --binary to disable.)
patching file src/src/m1qn3.f
(Stripping trailing CRs from patch; use --binary to disable.)
patching file src/blas/ddot.f
gfortran -fpic -c m1qn3.f -o m1qn3.o
gfortran -shared -c m1qn3.f -o libm1qn3.dll
gfortran -fpic -c ddot.f -o ddot.o
gfortran -shared -c ddot.f -o libddot.dll

Please help me analyze the cause of the error.

Thanks !

justinquinn

wudipao88

Okay, first that comes to mind is perhaps it is tripping on the fact that it is operating on something other than the C drive. Are you able to test checking out a copy of the repo to the C drive and running configuration from there?

Also, could you show me your .bashrc?

wudipao88

justinquinn

Do you mean to put ISSM on the C drive? Below is my .bashrc:

9cc38d87-871b-492c-a4ed-2997b35a399e.txt

6kB

justinquinn

wudipao88

Ah, try export ISSM_DIR=/e/ISSM in your .bashrc, start a new terminal, then try the PETSc installation again.

wudipao88

justinquinn

Thank you for your guidance. After changing the path in .bashrc, the installation of petsc is smoother than before. However, the installation is still not completely successful. Here is the installation feedback:

===============================================================================

Configuring PETSc to compile on your system

/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/base.py:221: SyntaxWarning: invalid escape sequence '\ '
prog = prog.replace('\ ',' ').replace('(','(').replace(')',')')
/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/base.py:221: SyntaxWarning: invalid escape sequence '('
prog = prog.replace('\ ',' ').replace('(','(').replace(')',')')
/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/base.py:221: SyntaxWarning: invalid escape sequence ')'
prog = prog.replace('\ ',' ').replace('(','(').replace(')',')')
/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/framework.py:205: SyntaxWarning: invalid escape sequence '['
self.help.title = 'Configure Help\n Comma separated lists should be given between [] (use [ ] in tcsh/csh)\n For example: --with-mpi-lib=[/usr/local/lib/libmpich.a,/usr/local/lib/libpmpich.a]\n Options beginning with --known- are to provide values you already know\n Options beginning with --with- indicate that you are requesting something\n For example: --with-clanguage=c++\n <prog> means a program name or a full path to a program\n For example:--with-cmake=/Users/bsmith/bin/cmake\n <bool> means a boolean, use either 0 or 1\n <dir> means a directory\n For example: --with-packages-download-dir=/Users/bsmith/Downloads\n For packages use --with-PACKAGE-dir=<dir> OR\n --with-PACKAGE-include=<dir> --with-PACKAGE-lib=<lib> OR --download-PACKAGE'
/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/setCompilers.py:225: SyntaxWarning: invalid escape sequence '\s'
strmatch = re.match('GNU Fortran\s+(.)\s+(\d+).(\d+)',output)
/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/setCompilers.py:240: SyntaxWarning: invalid escape sequence '\s'
strmatch = re.match('GNU Fortran\s+(.)\s+(\d+).(\d+)',output)
/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/setCompilers.py:255: SyntaxWarning: invalid escape sequence '\s'
strmatch = re.match('GNU Fortran\s+(.)\s+(\d+).(\d+)',output)
/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/setCompilers.py:270: SyntaxWarning: invalid escape sequence '\s'
strmatch = re.match('GNU Fortran\s+(.)\s+(\d+).(\d+)',output)
/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/types.py:242: SyntaxWarning: invalid escape sequence '\s'
MaxPathLength = re.compile('\nchar petsc_max_path_len\s?[\s?] = '+HASHLINESPACE+'\"([0-9]+)\"'+HASHLINESPACE+';').search(buf).group(1)
/e/ISSM/externalpackages/petsc/src/config/PETSc/options/petscclone.py:40: SyntaxWarning: invalid escape sequence '*'
self.addDefine('VERSION_BRANCH_GIT','"'+re.compile('* (.*)').search(o4).group(1)+'"')
/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/packages/MPI.py:213: SyntaxWarning: invalid escape sequence '\ '
self.mpiexec = self.mpiexec.replace(' ', '\ ').replace('(', '$').replace(')', '$').replace('\ -',' -')
/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/packages/MPI.py:569: SyntaxWarning: invalid escape sequence '.'
mpich_numversion = re.compile('\nconst char mpich_ver ='+HASHLINESPACE+'"([.0-9]+)"'+HASHLINESPACE+';').search(buf).group(1)
/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/packages/MPI.py:621: SyntaxWarning: invalid escape sequence '['
msmpi_version = re.compile('\nchar msmpi_hex[] = '+HASHLINESPACE+'\"([a-zA-Z0-9_])\"'+HASHLINESPACE+';').search(buf).group(1)
/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/packages/openblas.py:68: SyntaxWarning: invalid escape sequence '\s'
ver = re.match("\sOpenBLAS\s([0-9.]+)\s",ver).group(1)
/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/packages/SuiteSparse.py:64: SyntaxWarning: invalid escape sequence '\$'
ldflags+=" -L\$(INSTALL_LIB)"
/e/ISSM/externalpackages/petsc/src/config/BuildSystem/config/packages/sowing.py:57: SyntaxWarning: invalid escape sequence '('
ver = re.compile('bfort (sowing) release ([0-9]+).([0-9]+).([0-9]+)').match(output)
=============================================================================== ***** WARNING: Using default optimization C flags -g -O You might consider manually setting optimal optimization flags for your system with COPTFLAGS="optimization flags" see config/examples/arch--opt.py for examples =============================================================================== =============================================================================== ***** WARNING: Using default C++ optimization flags -g -O You might consider manually setting optimal optimization flags for your system with CXXOPTFLAGS="optimization flags" see config/examples/arch--opt.py for examples =============================================================================== =============================================================================== ***** WARNING: Using default FORTRAN optimization flags -g -O You might consider manually setting optimal optimization flags for your system with FOPTFLAGS="optimization flags" see config/examples/arch--opt.py for examples =============================================================================== TESTING: configureLibrary from config.packages.metis(config/BuildSystem/config/packages/metis.py:49) /e/ISSM/externalpackages/petsc/src/config/BuildSystem/retrieval.py:226: SyntaxWarning: invalid escape sequence '\;'
config.base.Configure.executeShellCommand('cd '+root+'; chmod -R a+r '+dirname+';find '+dirname + ' -type d -name "*" -exec chmod a+rx {} \;', log = self.log)
=============================================================================== Trying to download git://https://bitbucket.org/petsc/pkg-metis.git for METIS =============================================================================== =============================================================================== Configuring METIS with cmake; this may take several minutes =============================================================================== =============================================================================== Compiling and installing METIS; this may take several minutes =============================================================================== =============================================================================== Trying to download git://https://bitbucket.org/petsc/pkg-parmetis.git for PARMETIS =============================================================================== =============================================================================== Configuring PARMETIS with cmake; this may take several minutes =============================================================================== =============================================================================== Compiling and installing PARMETIS; this may take several minutes =============================================================================== =============================================================================== Trying to download git://https://bitbucket.org/petsc/pkg-fblaslapack for FBLASLAPACK =============================================================================== =============================================================================== Compiling FBLASLAPACK; this may take several minutes =============================================================================== =============================================================================== Trying to download git://https://bitbucket.org/petsc/pkg-scalapack for SCALAPACK =============================================================================== =============================================================================== Compiling and installing Scalapack; this may take several minutes =============================================================================== *******************************************************************************

UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details):

Error running make on SCALAPACK

makefile:15: arch-mswin-c-opt/lib/petsc/conf/petscvariables: No such file or directory
make[1]: *** No rule to make target 'arch-mswin-c-opt/lib/petsc/conf/petscvariables'. Stop.
gmakefile:5: arch-mswin-c-opt/lib/petsc/conf/petscvariables: No such file or directory
make: *** [GNUmakefile:17: arch-mswin-c-opt/lib/petsc/conf/petscvariables] Error 2
makefile:15: arch-mswin-c-opt/lib/petsc/conf/petscvariables: No such file or directory
make[1]: *** No rule to make target 'arch-mswin-c-opt/lib/petsc/conf/petscvariables'. Stop.
gmakefile:5: arch-mswin-c-opt/lib/petsc/conf/petscvariables: No such file or directory
make: *** [GNUmakefile:17: arch-mswin-c-opt/lib/petsc/conf/petscvariables] Error 2

justinquinn

wudipao88

There's something still off in your setup. On our end, for example, the output is like,

===============================================================================
             Configuring PETSc to compile on your system
===============================================================================
===============================================================================
      ***** WARNING: Using default optimization C flags -g -O
      You might consider manually setting optimal optimization flags for your system with
      COPTFLAGS="optimization flags" see config/examples/arch-*-opt.py for examples
===============================================================================
===============================================================================
      ***** WARNING: Using default C++ optimization flags -g -O
      You might consider manually setting optimal optimization flags for your system with
      CXXOPTFLAGS="optimization flags" see config/examples/arch-*-opt.py for examples
===============================================================================
===============================================================================
      ***** WARNING: Using default FORTRAN optimization flags -g -O
      You might consider manually setting optimal optimization flags for your system with
      FOPTFLAGS="optimization flags" see config/examples/arch-*-opt.py for examples
===============================================================================
===============================================================================
      Trying to download git://https://bitbucket.org/petsc/pkg-metis.git for METIS
===============================================================================
===============================================================================
      Configuring METIS with cmake; this may take several minutes
===============================================================================
[...]

without all the warnings about invalid escape sequences.

When you start up your terminal, does the prompt look like <USER>@<HOSTNAME> MINGW64 ~?

wudipao88

Yes, <USER>@<HOSTNAME> MINGW64 ~ is appeared when I start up MSYS2 MINGW64. Here is what I added to the .bashrc, is there any problem else?

## MATLAB # MATLAB_VER="R2024a" # Allows for easy resetting of MATLAB version added to path export MATLAB_PATH=$(cygpath -u $(cygpath -ms "/e/Program Files/MATLAB/${MATLAB_VER}")) export PATH="${MATLAB_PATH}/bin:${PATH}"

## ISSM # export ISSM_DIR=/e/ISSM export ISSM_DIR_WIN=$(cygpath -ms "${ISSM_DIR}") # Needed by MATLAB

Thanks !

justinquinn

wudipao88 Yeah, that all looks good. Do you have the ability to write to the C drive? If so, try checking out another copy of the repo there and see if compiling PETSc works. You will have to update ISSM_DIR in your .bashrc to the new location of course. I wonder if MSYS2 is having an issue with drives other than the default for some reason?

wudipao88

justinquinn I put the ISSM on the C drive and tried to install the external packages again, but when installing petsc there was still an error message of "invalid escape sequences", I really don't know what caused this installation error.😂

wudipao88

justinquinn I did not install Anaconda, is this related to my installation error?

justinquinn

wudipao88 Well that stinks. It was worth a try anyhow.

justinquinn

wudipao88 I'm going to try to do a fresh install of MSYS2 on a Windows VM to see if I can reproduce the error. Are you on Windows 10 or 11?

justinquinn

wudipao88 No, Anaconda is not required and can, in fact, cause a lot of issues with configuration/compiling.

wudipao88

justinquinn My operating system is Windows 10. Thanks a lot !

justinquinn

wudipao88 Good (bad?) news: I was able to recreate your issue with installing PETSc on Windows under MSYS2. Now I just have to sort out why it's happening there and not on the other machines. Will update you as soon as I have an answer.

justinquinn

wudipao88 Okay, so it may be an easy fix after all: looks like they removed gfortran from the base-devel package set in Pacman for MSYS2. You can install it with,

pacman -S mingw-w64-x86_64-gcc-fortran

You can effectively ignore the SyntaxWarning's from Python. This was introduced in Python 3.12. Let me know if this works for you.

justinquinn

wudipao88 Actually, I have the same issue now that you had with Scalapack under PETSc. Investigating further.

justinquinn

wudipao88

Alright, I have a solution for you! Note that this was all due to changes with the versions of GCC and Python that are current for pacman.

1) Run pacman -S python-setuptools mingw-w64-x86_64-gcc-fortran (I think you may have already installed gfortran)
2) Run git pull in $ISSM_DIR
3) Run the PETSc installation again

I think your issue with the patch in Triangle may be due to the fact that you had already run the installation. If you rm -rf $ISSM_DIR/externalpackages/triangle/install and run the install script again, it should work.

Output of the m1qn3 installation looks fine to me.

Let me know if this works for you and Happy New Year!