External Package Installation on Apple M1 Chip

lawrence-bird

Hi Justin/Mathieu:

Updating gfortran to an experimental ARM version for MacOS Monterey (gfortran 12-experimental for ARM, macOS 12 (Monterey)) did indeed allow me to successfully install PETSc. That said, I was able to install all external packages.

Moving on to the next step, ISSM appears to configure successfully, but is failing during make. I am receiving the following error:

ld: file not found: @rpath/libgcc_s.1.1.dylib for architecture arm64 clang: error: linker command failed with exit code 1 (use -v to see invocation)

It seems to be something to do with gcc; however, when I interrogate the gcc version on my system (below), it looks like there is a gcc version suitable for arm64:

$ which gcc /usr/bin/gcc

$ file /usr/bin/gcc /usr/bin/gcc: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e] /usr/bin/gcc (for architecture x86_64): Mach-O 64-bit executable x86_64 /usr/bin/gcc (for architecture arm64e): Mach-O 64-bit executable arm64e

$ gcc --version Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/4.2.1 Apple clang version 13.0.0 (clang-1300.0.29.30) Target: arm64-apple-darwin21.2.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

If you have any ideas on this, I'm all ears!

Thanks.

Lawrence

justinquinn

lawrence-bird
What do you get from find /usr -name libgcc_s*?

lawrence-bird

justinquinn

Hi Justin:

That command returns the following location: /usr/local/gfortran/lib/libgcc_s.1.1.dylib

Thanks.

Lawrence

justinquinn

lawrence-bird Okay, you may have to,

export LD_LIBRARY_PATH=/usr/local/gfortran/lib:${LD_LIBRARY_PATH}
export DYLD_LIBRARY_PATH=/usr/local/gfortran/lib:${DYLD_LIBRARY_PATH}

in your shell profile (the above assumes bash). You may be able to test this ahead of time by exporting (or otherwise setting these environment variables) directly in the terminal instance and recompiling.

lawrence-bird

justinquinn

Hi Justin - I implemented this, but the same error persists, unfortunately. I assume this would not affect the external package installations and I don't need to recompile those?

lawrence-bird

justinquinn

I am not sure if it's relevant where this error is occurring during the make process, but this is the full error:

ld: file not found: @rpath/libgcc_s.1.1.dylib for architecture arm64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make[3]: *** [libISSMCore.la] Error 1 make[2]: *** [all-recursive] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2

This is preceded by a number of warnings similar to this:

*** Warning: Linking the shared library libISSMCore.la against the *** static library /Users/lbir0005/ISSM/trunk/externalpackages/m1qn3/install/libddot.a is not portable! ld: warning: object file (./modules/FourierLoveCorex/.libs/lnb_param.o) was built for newer macOS version (12.1) than being linked (12.0)

justinquinn

lawrence-bird
I wouldn't think that this would affect any of the external packages, no.

justinquinn

lawrence-bird
Could you post the 10 lines or so before ld: file not found: @rpath/libgcc_s.1.1.dylib for architecture arm64?

lawrence-bird

justinquinn

Here's the full output once things start going awry:

*** Warning: Linking the shared library libISSMCore.la against the *** static library /Users/lbir0005/ISSM/trunk/externalpackages/m1qn3/install/libm1qn3.a is not portable! *** Warning: Linking the shared library libISSMCore.la against the *** static library /Users/lbir0005/ISSM/trunk/externalpackages/m1qn3/install/libddot.a is not portable! ld: warning: object file (./modules/FourierLoveCorex/.libs/lnb_param.o) was built for newer macOS version (12.1) than being linked (12.0) ld: warning: object file (./modules/FourierLoveCorex/.libs/model.o) was built for newer macOS version (12.1) than being linked (12.0) ld: warning: object file (./modules/FourierLoveCorex/.libs/util.o) was built for newer macOS version (12.1) than being linked (12.0) ld: warning: could not create compact unwind for ___model_MOD_earth_nlayers: register 28 saved somewhere other than in frame ld: warning: could not create compact unwind for ___model_MOD_earth_nlayers_viscoelas: register 75 saved somewhere other than in frame ld: warning: could not create compact unwind for ___model_MOD_model_init2: register 73 saved somewhere other than in frame ld: warning: object file (./modules/FourierLoveCorex/.libs/lovenb_sub.o) was built for newer macOS version (12.1) than being linked (12.0) ld: warning: could not create compact unwind for ___lovenb_sub_MOD_solution_nbloutput: registers 78 and 79 not saved contiguously in frame ld: warning: could not create compact unwind for ___util_MOD_ludcmp: register 24 saved somewhere other than in frame ld: warning: ld: warning: could not create compact unwind for ___util_MOD_derivs: registers 78 and 79 not saved contiguously in frameld: warning: could not create compact unwind for ___lovenb_sub_MOD_solution: registers 78 and 79 not saved contiguously in frameld: warning: object file (./modules/GiaDeflectionCorex/.libs/libISSMCore_la-ojrule.o) was built for newer macOS version (12.1) than being linked (12.0)ld: warning: object file (./modules/GiaDeflectionCorex/.libs/libISSMCore_la-freed.o) was built for newer macOS version (12.1) than being linked (12.0) ld: warning: ld: warning: object file (./modules/GiaDeflectionCorex/.libs/libISSMCore_la-distme.o) was built for newer macOS version (12.1) than being linked (12.0) object file (./modules/FourierLoveCorex/.libs/love_numbers.o) was built for newer macOS version (12.1) than being linked (12.0) ld: warning: object file (./modules/GiaDeflectionCorex/.libs/libISSMCore_la-pwise.o) was built for newer macOS version (12.1) than being linked (12.0) object file (./modules/GiaDeflectionCorex/.libs/libISSMCore_la-qwise.o) was built for newer macOS version (12.1) than being linked (12.0)ld: warning: ld: warning: object file (./modules/GiaDeflectionCorex/.libs/libISSMCore_la-what0.o) was built for newer macOS version (12.1) than being linked (12.0) ld: warning: could not create compact unwind for _what0_: registers 78 and 79 not saved contiguously in frame ld: warning: could not create compact unwind for ___lovenb_sub_MOD_boundary_conditions_matrix: register 77 saved somewhere other than in frame ld: warning: object file (./modules/GiaDeflectionCorex/.libs/libISSMCore_la-stot.o) was built for newer macOS version (12.1) than being linked (12.0) ld: warning: could not create compact unwind for _stot_: registers 72 and 73 not saved contiguously in frame could not create compact unwind for _love_numbers_: registers 27 and 28 not saved contiguously in frame ld: warning: could not create compact unwind for _lnb_setup_: registers 27 and 28 not saved contiguously in frame ld: warning: could not create compact unwind for _distme_: register 75 saved somewhere other than in frame ld: warning: dylib (/Users/lbir0005/ISSM/trunk/externalpackages/petsc/install/lib/libmpifort.dylib) was built for newer macOS version (12.1) than being linked (12.0) ld: warning: dylib (/usr/local/gfortran/lib/gcc/aarch64-apple-darwin21/12.0.0/../../..//libgfortran.5.dylib) was built for newer macOS version (12.1) than being linked (12.0) ld: warning: dylib (/usr/local/gfortran/lib/gcc/aarch64-apple-darwin21/12.0.0/../../..//libquadmath.0.dylib) was built for newer macOS version (12.1) than being linked (12.0) ld: file not found: @rpath/libgcc_s.1.1.dylib for architecture arm64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make[3]: *** [libISSMCore.la] Error 1 make[2]: *** [all-recursive] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2

justinquinn

lawrence-bird
Hi Lawrence,
The warnings may look alarming but do not indicate an actual failure. I think what might be going on is that the binaries of the this copy of gfortran are functional but the /usr/local/gfortran/lib/libgcc_s.1.1.dylib may not be compiled with ARM as the target. To be perfectly honest, I don't know what is trying to linke against libgcc_s. What do you get from ldd /usr/local/bin/gfortran? Also, what does your configuration look like?

lawrence-bird

justinquinn

Hi Justin:

ldd doesn't appear to work (I'm not sure if it's all Mac's or just these new systems), but apparently otool -L is the equivalent. Output from this:

$ otool -L /usr/local/bin/gfortran /usr/local/bin/gfortran: /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.0.0)

Perhaps I will try rolling back my gfortran version to the one listed by Mathieu and see if that works on Monterey (as well as it's intended MacOS 11).

justinquinn

lawrence-bird
Right, ldd aliases to otool-classic in my case.

Okay, the output looks the same as mine. You might try running down the dependency trail by running otool -L on each of the libraries that are listed to see if they, in turn, load libgcc_s.1.1.dylib. That said, I suspect it is something else that is linking to this library. It may, after all, be one of the external package libraries, so you may have to check those too with otool.

Try rolling back your gfortran version though first and let's see if that resolves it.

lawrence-bird

justinquinn

Hi Justin:

Rolling back the gfortran version didn't resolve the issue. I've also dug around the external packages using otool -L and can't find any of those that link to libgcc_s.1.1.dylib.

The only thing that I notice in the config.log file is a "fatal error" that occurs a few times, although it dosesn't result in the configure process failing:

configure:3883: checking how to run the C preprocessor configure:3914: gcc -E conftest.c configure:3914: $? = 0 configure:3928: gcc -E conftest.c conftest.c:9:10: fatal error: 'ac_nonexistent.h' file not found #include <ac_nonexistent.h> ^~~~~~~~~~~~~~~~~~ 1 error generated. configure:3928: $? = 1 configure: failed program was: | /* confdefs.h */ | #define PACKAGE_NAME "Ice Sheet System Model (ISSM)" | #define PACKAGE_TARNAME "issm" | #define PACKAGE_VERSION "4.19" | #define PACKAGE_STRING "Ice Sheet System Model (ISSM) 4.19" | #define PACKAGE_BUGREPORT "issm@jpl.nasa.gov" | #define PACKAGE_URL "http://issm.jpl.nasa.gov" | /* end confdefs.h. */ | #include <ac_nonexistent.h>

I don't think this is related, but thought I would share it in case it sheds any light on anything.

Thanks.

Lawrence

justinquinn

lawrence-bird
Hi Lawrence,
I'm sorry, I'm running out of ideas on what might be the issue here. It's really hard for me to test on my end without access to an M1 machine. Are you getting the same error as before with the rolled back copy of gfortran?

I can confirm that the fatal errors in the config.log are not an issue:

$ grep "fatal error" config.log 
config.log:155:conftest.c:9:10: fatal error: 'ac_nonexistent.h' file not found
config.log:174:conftest.c:9:10: fatal error: 'ac_nonexistent.h' file not found
config.log:256:gfortran: fatal error: no input files
config.log:261:gfortran: fatal error: no input files
config.log:297:gfortran: fatal error: no input files
config.log:302:gfortran: fatal error: no input files
config.log:545:conftest.cpp:23:10: fatal error: 'ac_nonexistent.h' file not found
config.log:578:conftest.cpp:23:10: fatal error: 'ac_nonexistent.h' file not found

That's from a successful config and compilation.

lawrence-bird

justinquinn
Hi Justin:

No worries - this is a strange one.

I m happy to report that I was able to get ISSM to compile! But I've now got a new issue. On the compilation front, this was my process:

Grabbed a clean version from the trunk
Downloaded the gfortran 11-experimental-2 for ARM Big Sur (macOS 11) version Mathieu noted above
Installed all external packages (noting the updated to LIBTOOL_VER=2.46 for autotools
Ran autoreconf -ivf at $ISSM_DIR
Installed numpy / scipy using homebrew
Installed pyenv using homebrew (to allow me to get a dev copy of Python)
Installed python 3.9-dev using pyenv
Specified the --with-fortran-lib in the configure script to point to the location of gfortran that I'd downloaded. This step was important as another gcc version is installed using homebrew as a dependancy for numpy. Without specifying the fortran location, it would try and use the gcc location in homebrew (based, I presume, on it's location in the $PATH), and it would fail. Interestingly, the gcc version downloaded as a dependancy for numpy is that same version I was originally trying (for Monterey).
Specified the python directory (in pyenv) and the python version (3.9) in the configure script.
./configure
make
make install

This appears to result in a successful compilation!

However, when I try and run the model via Python, I am returned a Segmentation fault: 11. After launching Python, I am entering the following (before be returned the seg fault):

import sys sys.path.append('/Users/lbir0005/ISSM/trunk/bin/') sys.path.append('/Users/lbir0005/ISSM/trunk/lib/') from issmversion import issmversion

I was able to successfully build ISSM with Matlab on a different machine that I am waiting to get remote access right for.

At this point, I am just being stubborn about trying to get it working on my laptop with Python.

Cheers,

Lawrence

justinquinn

lawrence-bird
Nice! Glad to hear you got through compilation at least. The only other suggestions I could make is to make sure you incorporate the following options into your configuration (adjusting the paths as needed, of course):

--with-cxxoptflags="-D_DO_NOT_LOAD_GLOBALS_ -g -O2 -fPIC -std=c++11" \
--with-python-version=3.9 \
--with-python-dir=/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9 \
--with-python-numpy-dir=/usr/local/lib/python3.9/site-packages/numpy

I've included the -D_DO_NOT_LOAD_GLOBALS_ flag as a test because you did not receive an error related to it, but we have seen issues with more recent GCC versions that this has been a short term fix for.

Please do let us know if you get everything compiled and running with the Python API as you're in uncharted territory and it will be of use to others. Thanks!

lawrence-bird

justinquinn

Thanks Justin - I'll try adding the -D_DO_NOT_LOAD_GLOBALS_, just in case.

Two (silly) questions around working with Python and ISSM:

When I tried to compile using Python Version 3.9.10 (Installed along side numpy with homebrew), I received the error: Python.h not found error. Reading the thread here, I installed Python 3.9-dev using pyenv and things were resolved. Is this the correct way of dealing with this, or is there potentially something else resulting in this error?
Do I need to work with a Python IDE (e.g., IPython), or should I be able to get things rolling by working directly in the terminal with Python?

Thanks,

Lawrence

justinquinn

lawrence-bird
(1) Yes, as far as I know, newer versions of macOS do not include the Python header file, so you have to install your own copy. I personally use Homebrew, but I expect that other installations work just fine. (If you're interested in how we find Python.h given --with-python-version and --with-python-dir during configuration, check out m4/issm_options.m4).
(2) You do not need an IDE or GUI unless you wish to get into plotting. All of the computational features will be available to you. You might try pinging @bdefleurian for advice on generating plots under Python.

lawrence-bird

justinquinn
Hi Justin:

Good news! After a few days away, I simply made a clean build of ISSM and I'm now up and running with Python3.9 on my M1 machine!

Using PyCharm, I'm able to walk through the Square Ice Shelf model and generate the resultant basic plot. I've also been able to successfully walk through the majority of the mesh example.

One thing I wanted to confirm - I understand that using ISSM within Python is not fully supported. Does this mean that there are core components of the model that simply don't work in Python, or just that there are fewer users/folks maintaining this? This comes up in the mesh tutorial, when calling exptool() - I can't seem to locate this function within the Python build. Just wondering if this is a result of some of these tools not being available in Python, or something more sinister.

Ultimately, I need to make a decision on which route to go with ISSM (Python - via my laptop, or MATLAB - via remote access to a machine with an Intel processor). I would be interested to explore using the Python version, perhaps with some support from R for pre-/post-processing (simply because I am much more familiar with this framework), but don't want to short change myself too much on built-in available features.

Thanks.

Lawrence

justinquinn

lawrence-bird
Hi Lawrence,
That is great news! Thanks for updating us and this thread.

There are indeed some missing components of the Python API, and this does have to do with maintainability. Most of our core developers use the MATLAB API, so when new modules are developed, they sometimes do not get translates to Python right away. As well, modules that have plotting capabilities written for MATLAB are not going to have an easy or direct translation to multiple Python plotting libraries, so we have left it up to the user to decided which library works best for them. In the future, we may decide to go with a default plotting library, but there are no plans currently for this.

If you wanted to go about patching missing functionality/modules, I would go through the following steps:

Check if the missing function is a MATLAB builtin by searching the MATLAB documentation. In many cases there is an identical or similar function in NumPy.

Check if the function call is a module, and if so, if the Python version is missing. For example, from $ISSM_DIR, I might run,

$ find src -name exptool.*
src/m/exp/exptool.m

which shows that the exptool module indeed has not been translated yet. If you were to make such translations I think we would be happy to have you on as contributing developer and get you access to the development branch.

I understand not wanting to commit to the "lesser" of the available APIs, but I'm not sure there is really a huge gap in functionality besides the plotting capabilities. Indeed, most of what ISSM does is actually handled by the C++ core while the APIs are mostly for file I/O, model setup, result retrieval, and plotting. I'd be happy to help come up with a rough number of missing Python modules if you think that would help you make your decision.

Best,
Justin

lawrence-bird

justinquinn
Hi Justin:

Thanks for the summary here - this is what I suspected. I guess my biggest concern is running into snags down the road and having a limited community support for the Python API. That being said, I think it would be really helpful to have an idea how many/which modules may be missing from the Python API. If I could identify if these are generally pretty niche and I am unlikely to require them, that would probably sway me.

Any help you can provide on this would be great!

Thanks.

Lawrence

« Previous Page Next Page »