External Package Installation on Apple M1 Chip

justinquinn
Thanks Justin. I persevered with the Python setup for a couple additional tutorials. I'm running into some issues with the Inversion tutorial, but I am not convinced its due to lack of python modules. I think it stems back to the build on my M1 machine.

When running md=solve(md, 'Stressbalance') with md.inversion.iscontrol=1, the process seems to stall without actually returning any solutions. Below is the full output I get after calling this solve() command:

============================================================
Binary file Square.outbin not found
This typically happens when the run crashed.
Please check for error messages above or in the outlog ============================================================
WARNING: Square.outbin does not exist
[0]PETSC ERROR: ------------------------------------------------------------------------
[0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range
[0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
[0]PETSC ERROR: or see https://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind
[0]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors
[0]PETSC ERROR: configure using --with-debugging=yes, recompile, link, and run
[0]PETSC ERROR: to get more information on the crash.
[0]PETSC ERROR: --------------------- Error Message --------------------------------------------------------------
[0]PETSC ERROR: Signal received
[0]PETSC ERROR: See https://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.15.2, Jul 10, 2021
[0]PETSC ERROR: /Users/lbir0005/ISSM/trunk/bin/issm.exe on a named MU00186957X by lbir0005 Fri Feb 11 15:49:29 2022
[0]PETSC ERROR: Configure options --prefix=/Users/lbir0005/ISSM/trunk/externalpackages/petsc/install --PETSC_DIR=/Users/lbir0005/ISSM/trunk/externalpackages/petsc/src --CFLAGS=-Wno-error=implicit-function-declaration --FFLAGS=-fallow-argument-mismatch --LDFLAGS=-Wl,-no_compact_unwind --with-debugging=0 --with-valgrind=0 --with-x=0 --with-ssl=0 --with-pic=1 --download-fblaslapack=1 --download-mpich=1 --download-metis=1 --download-parmetis=1 --download-scalapack=1 --download-mumps=1 --download-zlib=1
[0]PETSC ERROR: #1 User provided function() at unknown file:0
[0]PETSC ERROR: Run with -malloc_debug to check if memory corruption is causing the crash.
application called MPI_Abort(MPI_COMM_WORLD, 59) - process 0
Traceback (most recent call last):
File "/opt/homebrew/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/code.py", line 90, in runcode
exec(code, self.locals)
File "<input>", line 12, in <module>
File "/Users/lbir0005/ISSM/trunk/bin/solve.py", line 164, in solve
md = loadresultsfromcluster(md)
File "/Users/lbir0005/ISSM/trunk/bin/loadresultsfromcluster.py", line 68, in loadresultsfromcluster
remove(md.private.runtimename + '.tar.gz')
AttributeError: 'NoneType' object has no attribute 'private'

It looks like this has something to do with Petsc / MPI - I've included my ./configure options below in case there is anything here that you suggest changing.

./configure \
--prefix="$ISSM_DIR" \
--with-triangle-dir="$ISSM_DIR/externalpackages/triangle/install" \
--with-cxxoptflags="-D_DO_NOT_LOAD_GLOBALS_ -g -O2 -fPIC -std=c++11" \
--with-python-dir="/opt/homebrew/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9" \
--with-python-version=3.9 \
--with-python-numpy-dir="/opt/homebrew/Cellar/numpy/1.22.1/lib/python3.9/site-packages/numpy/core/include/numpy" \
--with-fortran-lib="-L/usr/local/gfortran/lib -lgfortran" \
--with-mpi-include="$ISSM_DIR/externalpackages/petsc/install/include" \
--with-mpi-libflags="-L$ISSM_DIR/externalpackages/petsc/install/lib -lmpi -lmpicxx -lmpifort" \
--with-petsc-dir="$ISSM_DIR/externalpackages/petsc/install" \
--with-scalapack-dir="$ISSM_DIR/externalpackages/petsc/install/" \
--with-mumps-dir="$ISSM_DIR/externalpackages/petsc/install/" \
--with-blas-lapack-dir="$ISSM_DIR/externalpackages/petsc/install" \
--with-metis-dir="$ISSM_DIR/externalpackages/petsc/install/" \
--with-m1qn3-dir="$ISSM_DIR/externalpackages/m1qn3/install" \
--with-numthreads=2

I suspect it's some Petsc intricacies, but I'm all ears is you have any ideas!

Cheers,

Lawrence

justinquinn
Hi Justin:

I was largely walking through the runme.py script from the Inversion tutorial, with a few tweaks. I've posted the script below - The failure happens on the last line (when solving the inversion). Turning the inversion off (md.inversion.iscontrol = 0) removes the error. I've been having an issue installing the NetCDF4 library, so I haven't been using the export_NetCDF command to save the model to file.

import sys
import numpy as np
import os
sys.path.append('/Users/lbir0005/ISSM/trunk/bin/')
sys.path.append('/Users/lbir0005/ISSM/trunk/lib/')
#
from model import *
from triangle import *
from setmask import setmask
from parameterize import parameterize
from setflowequation import setflowequation
from generic import generic
from solve import solve
from plotmodel import plotmodel
#from export_netCDF import export_netCDF
from m1qn3inversion import m1qn3inversion
from verbose import verbose
#from loadmodel import loadmodel
from cuffey import cuffey'
#
#Set working directory to Inversion to make use of files
os.chdir("/Users/lbir0005/ISSM/trunk/examples/Inversion")
#
steps = [1, 2, 3, 4]
Clims = [1.3 * 1e8, 1.9 * 1e8]
#
if 1 in steps:
#Generate observations
md = model()
md = triangle(md, 'DomainOutline.exp', 100000)
md = setmask(md, 'all', '')
md = parameterize(md, 'Square.py')
md = setflowequation(md, 'SSA', 'all')
md.cluster = generic('np', 1)
md = solve(md, 'Stressbalance')
plotmodel(md, 'data', md.materials.rheology_B)
plotmodel(md, 'data', md.results.StressbalanceSolution.Vel)
#plotmodel(md, 'axis#all', 'tight', 'data', md.materials.rheology_B, 'caxis', Clims, 'title', '"True" B', 'data', md.results.StressbalanceSolution.Vel, 'title', '"observed velocities"')
#export_netCDF(md, 'model1.nc')
#
if 2 in steps:
#Modify rheology, now constant
#md = loadmodel('model1.nc')
md.materials.rheology_B[:] = 1.8 * 1e8
#
#results of previous run are taken as observations
md.inversion = m1qn3inversion()
md.inversion.vx_obs = md.results.StressbalanceSolution.Vx
md.inversion.vy_obs = md.results.StressbalanceSolution.Vy
md.inversion.vel_obs = md.results.StressbalanceSolution.Vel
#
md = solve(md, 'Stressbalance')
plotmodel(md, 'data', md.materials.rheology_B)
plotmodel(md, 'data', md.results.StressbalanceSolution.Vel)
#plotmodel(md, 'axis#all', 'tight', 'data', md.materials.rheology_B, 'caxis', Clims, 'title', 'B first guess', 'data', md.results.StressbalanceSolution.Vel, 'title', 'modeled velocities')
#
if 3 in steps:
#invert for ice rigidity
#md = loadmodel('model2.nc')
#
#Set up inversion parameters
maxsteps = 20
md.inversion.iscontrol = 1
md.inversion.control_parameters = ['MaterialsRheologyBbar']
md.inversion.maxsteps = maxsteps
md.inversion.cost_functions = [101]
md.inversion.cost_functions_coefficients = np.ones((md.mesh.numberofvertices, 1))
md.inversion.min_parameters = cuffey(273) * np.ones((md.mesh.numberofvertices, 1))
md.inversion.max_parameters = cuffey(200) * np.ones((md.mesh.numberofvertices, 1))
#
#Go solve!
#md.verbose = verbose(0)
md.verbose.solution = True
md = solve(md, 'Stressbalance')

Thanks,

Lawrence