It's a clean install (on docker actually). I've even looked at the jenkins build script and as much of the source code as I can dig into, but exhausted most options already 😅

Can you post your config.log I can look if there is something fishy in there. Just to confirm you are on the trunk version of ISSM not on the development branch?

It's a bit big so here's a link to the config.log file. And yes, I should be on the trunk version.

Hi guys, sorry that I am just now getting the opportunity to take a look at this.

On Debian, using Python 3.7, everything compiles just fine. There is a runtime crash with "ImportError: dynamic module does not define init function", which may be due to the fact that /usr/bin/python -> /usr/bin/python2 on this particular machine and --with-python-version is not actually being honored. That being said, I am currently experimenting to see if which python is the issue here.

Of note, I am working out of the development branch, so I will start diffing config files to see if there is a disconnect on our end.

In the meantime, @weiji14, could you try installing 3.7 (and the corresponding numpy package) with apt and see if you can get ISSM to at least compile? This would at least narrow down the issue(s) we are experiencing.

After linking /usr/bin/python to /usr/bin/python3, everything compiles and runs as expected. So, I think we can assume that, after the next merge from trunk-jpl/ to trunk/, the issue is not internal. That said, I will work on and commit the following fixes:

  1. Compilation should honor Python version supplied to option --with-python-version.
  2. Polling Python.h with python3 should not error out on the syntax error noted above.

Please let me know if I have missed anything and I will add it to this list. As well, @weiji14 let me know what if any progress you are able to make.

Okay, I have fixed (1) and committed to trunk-jpl/. I realized after the fact that (2) was a non-issue: what was happening was that I was compiling with flags for Python 3, but my test deck was running python, which was sym linked to python2. In all, I would advise running a copy of the development branch until we can push to the production branch (trunk/), which will happen soon (I am wrapping another, larger fix). We'll post here to the forum when that has been done.

In the meantime, @weiji14, could you try installing 3.7 (and the corresponding numpy package) with apt and see if you can get ISSM to at least compile? This would at least narrow down the issue(s) we are experiencing.

Using the apt provided numpy doesn't work on Ubuntu (and also on Debian). I've made sure to compile on a virtual environment using Python 3 (tested versions 3.6, 3.7, 3.8) so that Python 2 is not used but no luck... I think you're right that there's some symlink of python -> python2 that is messing things up.

I would advise running a copy of the development branch until we can push to the production branch (trunk/)

Don't think that I can access the dev branch (assuming that it's https://issm.ess.uci.edu/svn/issm/issm/trunk-jpl?). So I'll just wait things to get into the main trunk 😃

    weiji14 You may have already done the following, but just in case, this is the methodology I used to install Python 2 and 3 with all required packages under Debian:

    # Install development versions
sudo apt-get install python-dev python3-dev 

# Install minimal versions
sudo apt-get install python-minimal python3-minimal 

# Install pip
sudo apt-get install python-pip python3-pip

# Install required packages
sudo su
python -m pip install numpy scipy matplotlib nose
python3 -m pip install numpy scipy matplotlib nose

    And, yes: if you compile for Python 3 via --with-python-version, you will have to manually run python3* or create a new sym link if running python executes version 2.*.

      Thanks for listing down all the dependencies @justinquinn. I was only missing the python3-minimal dependency, but installing it still resulted in the same pyconfig.h: No such file or directory error. I've also symlinked python to python3 and compilation still fails. Might be time to try out the development branch, could you provide me with instructions to clone trunk-jpl if possible?

      5 days later

      Ok, I've updated to using revision 24686 (02 Apr 2020) on trunk and think I've resolved the problem after some tweaking. I've set both --with-python-dir and --with-python-numpy-dir to use $CONDA_DIR (instead of /usr) and that resolves the pyconfig.h not found error. The key was probably to be consistent (i.e. don't mix stuff between virtual environments).

      There was then another error about ld not able to find python3.8m which I resolved by patching the configure file to use python3.8 instead using the below command:

      sed -i 's/-lpython${PYTHON_VERSION}m/-lpython${PYTHON_VERSION}/g' $ISSM_DIR/configure

      Specifically, that patches the below code block in configure:

      if test "x${PYTHON_MAJOR}" == "x3"; then
	PYTHONLIB="-L${PYTHON_ROOT}/lib -lpython${PYTHON_VERSION}m"
else
	PYTHONLIB="-L${PYTHON_ROOT}/lib -lpython${PYTHON_VERSION}"

      Just wondering why the 'm' version is used for python 3.X and not python 2.x?

      To be honest I am not completely sure. The point is that when I migrated to Python3, the installation of python-dev (and hence the location of Python.h) decided that I needed the m version of this new Python. So I added those lines as I expect that this was a standard thing. So I supose that on your 3.8 your Python.h does not reside in python3.8m but in python3.8?
      So short answer is that It worked for me without the m in 2.x versions and broke in 3.x. I will look into it to make a wiser test.

        17 days later

        If you write print() function in a program and someone using Python 2.x tries to run it, they will get an error. To avoid this, it is a good practice to import print function :

        from __future__ import print_function

        The from future import print_function ; to bring the print function from Python 3 into Python 2.x. Now your code works on both Python 2.x and Python 3.x . The future statements need to be near the top of the file because they change fundamental things about the language, and so the compiler needs to know about them from the beginning.

          harronmarsh Hi, I am not experiencing this issue,

          $ python
...
>>> import sys
>>> sys.version_info
sys.version_info(major=2, minor=7, micro=16, releaselevel='final', serial=0)
>>> print "Hello, World!"
Hello, World!
>>> print("Hello, World!")
Hello, World!
>>> msg = "Hello, World!"
>>> print msg
Hello, World!
>>> print(msg)
Hello, World!
            2 years later

            weiji14 checking for python header file Python.h... configure: error: Python.h not found, locate this file and contact ISSM developers

            Most of the time these are dependency-issues. Python.h is used by GNU Compiler Collection (gcc) to build applications. You need to install a package called python-dev for building Python modules, extending the Python interpreter or embedding Python in applications. You encounter "Python.h: No such file or directory" error while trying to build a shared library using the file extension of another language ( e.g. 'C' ). If you are trying to build a shared library using the file extension of another language, you need to install the correct development version of Python.

            Reason for this error:

            • You haven't properly installed the header files and static libraries for python dev.
            • Also, sometimes include files might not be default in the include path.

            How to solve this error:

            • Install the missing files and libraries.
            • Include Path and Library.
            • Finally, Compile it.