Best practices for updating ISSM Version

lawrence-bird

Hi folks:

I've been reluctant to update my ISSM since May 2022 because of the intricacies of compiling on M1. However, I think we have a better sense of how this should go now, and it seems about time for me to update!

What is the recommended approach to update to the latest release of ISSM? Would it be worth me attempting a completely fresh build given that all external package versions etc. are from ₁₀ months ago, or should I be able to just checkout the latest source code and re-compile/make?

Thanks!

mathieumorlighem

Hi @lawrence-bird

I do not recommend reinstalling external packages. We very rarely change them. A fresh build should be a last resort. Try simply updating and recompiling the code. Here are the steps:

update the code

cd $ISSM_DIR
svn update

you will see a list of files being updating (U), deleted (D), or added (A). If you see a Makefile.am or Issm_option.m4 in this list, you will need to reconfigure. If you are not sure, do it anyway, it can't hurt:

autoreconf -ivf
./configure.sh

if any of the files in the list include a c++ file (*.cpp) or a header file (*.h), or if you did step 2 above, you will need to recompile the code:

make -j 8 install

(assuming you have 8 cores on your machine, you can increase or decrease this number accordingly).

Best
Mathieu

lawrence-bird

Thanks mathieumorlighem! I'll let you know if I have any issues.

rlaw

Hi @lawrence-bird ,

How was this for you? I might also update on a build I was happy to get going in the first place!

Best,
Rob

lawrence-bird

Hi rlaw & @mathieumorlighem,

I finally got around (worked up the courage) to give this a whirl today. I broke it 🙁

I followed those steps outlined above. Everything seemed to go as planned, and ISSM built successfully. However, when I try and run a rheology inversion, I get a Segmentation Fault related to PETSc that occurs after the first iteration:

Cost function f(x)   | Gradient norm |g(x)| |  List of contributions
____________________________________________________________________
f(x) =     23.44563  |         9.593029e-09 |     18.14913     5.294879  0.001621468
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range
[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[0]PETSC ERROR: or see https://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors
[0]PETSC ERROR: configure using --with-debugging=yes, recompile, link, and run 
[0]PETSC ERROR: to get more information on the crash.
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Signal received
[0]PETSC ERROR: See https://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.15.2, Jul 10, 2021 
[0]PETSC ERROR: /Users/lbir0005/issm/trunk//bin/issm.exe on a  named MU00186957X by lbir0005 Wed Feb  1 16:24:12 2023
[0]PETSC ERROR: Configure options --prefix=/Users/lbir0005/issm/trunk/externalpackages/petsc/install --PETSC_DIR=/Users/lbir0005/issm/trunk/externalpackages/petsc/src --CFLAGS=-Wno-error=implicit-function-declaration --FFLAGS=-fallow-argument-mismatch --LDFLAGS=-Wl,-no_compact_unwind --with-debugging=0 --with-valgrind=0 --with-x=0 --with-ssl=0 --with-pic=1 --download-fblaslapack=1 --download-mpich=1 --download-metis=1 --download-parmetis=1 --download-scalapack=1 --download-mumps=1 --download-zlib=1
[0]PETSC ERROR: #1 User provided function() at unknown file:0
[0]PETSC ERROR: Run with -malloc_debug to check if memory corruption is causing the crash.
[1]PETSC ERROR: ------------------------------------------------------------------------
[1]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range
[1]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[1]PETSC ERROR: or see https://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
[1]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors
[1]PETSC ERROR: configure using --with-debugging=yes, recompile, link, and run 
[1]PETSC ERROR: to get more information on the crash.
[1]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[1]PETSC ERROR: Signal received
[1]PETSC ERROR: See https://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
[1]PETSC ERROR: Petsc Release Version 3.15.2, Jul 10, 2021 
[1]PETSC ERROR: /Users/lbir0005/issm/trunk//bin/issm.exe on a  named MU00186957X by lbir0005 Wed Feb  1 16:24:12 2023
[1]PETSC ERROR: Configure options --prefix=/Users/lbir0005/issm/trunk/externalpackages/petsc/install --PETSC_DIR=/Users/lbir0005/issm/trunk/externalpackages/petsc/src --CFLAGS=-Wno-error=implicit-function-declaration --FFLAGS=-fallow-argument-mismatch --LDFLAGS=-Wl,-no_compact_unwind --with-debugging=0 --with-valgrind=0 --with-x=0 --with-ssl=0 --with-pic=1 --download-fblaslapack=1 --download-mpich=1 --download-metis=1 --download-parmetis=1 --download-scalapack=1 --download-mumps=1 --download-zlib=1
[1]PETSC ERROR: #1 User provided function() at unknown file:0
[1]PETSC ERROR: Run with -malloc_debug to check if memory corruption is causing the crash.
application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0
application called MPI_Abort(MPI_COMM_WORLD, 59) - process 1

I haven't dug into this too much yet - I thought I'd see if you had any ideas off the bat. I wondered if it could be something to do with the fact that I've had various OS updates over the last year since I installed the external packages (and therefore, perhaps I should be updating these)? I'm happy to give a fresh build a go (given that I've broken it anyway!) if you think it might be useful.

I notice in the makefile that various build/host/os flags are updated from darwin21.4.0 to darwin21.6.0 (and PETSc would have been compiled again darwin21.4.0). Not sure if this indicates any mismatches that might be an issue.

Thoughts welcome and appreciated!

Lawrence

mathieumorlighem

Hi @lawrence-bird

This is bad, and not easy to debug... it is a segmentation fault. Even though it says "PETSc" it is probably a bug in ISSM. Can you save your model right before you call md=solve(md,... and send it to me by email? I will look into it.
All the best
Mathieu