Error while loading share libraries: libpetsc

mathieumorlighem

if libpetsc.so cannot be found at run time, that's because you need to add:

fprintf(fid,'source $ISSM_DIR/etc/environment.sh\n');

to make sure PETSc is added to your path.

adamjhepburn

mathieumorlighem

After adding that the queue script now sources the environment.sh script but it still happens... I have tried changing them to explicit paths in the queue script too just to check everything was correct there,

thanks,
Adam

mathieumorlighem

you should add

ldd <PATH to ism.exe>/issm.exe

to the queueing script and see what comes out

adamjhepburn

mathieumorlighem here is the output

` linux-vdso.so.1 => (0x00007ffe882f5000)
libISSMCore.so => /users/gg4ahe/issm/lib/libISSMCore.so (0x00007f96ec105000)
libISSMOverload.so => /users/gg4ahe/issm/lib/libISSMOverload.so (0x00007f96ece93000)
libstdc++.so.6 => /opt/apps/testapps/el7/software/staging/GCCcore/12.2.0/lib64/libstdc++.so.6 (0x00007f96ebee2000)
libpetsc.so.3.21 => /users/gg4ahe/issm/externalpackages/petsc/install/lib/libpetsc.so.3.21 (0x00007f96e9878000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f96e9674000)
libparmetis.so => /users/gg4ahe/issm/externalpackages/petsc/install/lib/libparmetis.so (0x00007f96ece20000)
libscalapack.so.2.2 => /users/gg4ahe/issm/externalpackages/petsc/install/lib/libscalapack.so.2.2 (0x00007f96e8f9d000)
libmetis.so => /users/gg4ahe/issm/externalpackages/petsc/install/lib/libmetis.so (0x00007f96ecd85000)
libmkl_intel_lp64.so.2 => /opt/apps/testapps/el7/software/staging/imkl/2022.2.1/mkl/2022.2.1/lib/intel64/libmkl_intel_lp64.so.2 (0x00007f96e7d82000)
libmkl_sequential.so.2 => /opt/apps/testapps/el7/software/staging/imkl/2022.2.1/mkl/2022.2.1/lib/intel64/libmkl_sequential.so.2 (0x00007f96e635a000)
libmkl_core.so.2 => /opt/apps/testapps/el7/software/staging/imkl/2022.2.1/mkl/2022.2.1/lib/intel64/libmkl_core.so.2 (0x00007f96e1f85000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f96e1d69000)
libmpicxx.so.12 => /opt/apps/testapps/el7/software/staging/impi/2021.7.1-intel-compilers-2022.2.1/mpi/2021.7.1/lib/libmpicxx.so.12 (0x00007f96e1b49000)
libmpifort.so.12 => /opt/apps/testapps/el7/software/staging/impi/2021.7.1-intel-compilers-2022.2.1/mpi/2021.7.1/lib/libmpifort.so.12 (0x00007f96e1795000)
libmpi.so.12 => /opt/apps/testapps/el7/software/staging/impi/2021.7.1-intel-compilers-2022.2.1/mpi/2021.7.1/lib/release/libmpi.so.12 (0x00007f96dff60000)
libifcore.so.5 => /opt/apps/testapps/el7/software/staging/intel-compilers/2022.2.1/compiler/2022.2.1/linux/compiler/lib/intel64_lin/libifcore.so.5 (0x00007f96dfe28000)
libifport.so.5 => /opt/apps/testapps/el7/software/staging/intel-compilers/2022.2.1/compiler/2022.2.1/linux/compiler/lib/intel64_lin/libifport.so.5 (0x00007f96ecd59000)
libgfortran.so.5 => /opt/apps/testapps/el7/software/staging/GCCcore/12.2.0/lib64/libgfortran.so.5 (0x00007f96dfb5c000)
libquadmath.so.0 => /opt/apps/testapps/el7/software/staging/GCCcore/12.2.0/lib64/libquadmath.so.0 (0x00007f96ecd11000)
libm.so.6 => /lib64/libm.so.6 (0x00007f96df85a000)
librt.so.1 => /lib64/librt.so.1 (0x00007f96df652000)
libgcc_s.so.1 => /opt/apps/testapps/el7/software/staging/GCCcore/12.2.0/lib64/libgcc_s.so.1 (0x00007f96eccef000)
libc.so.6 => /lib64/libc.so.6 (0x00007f96df284000)
libimf.so => /opt/apps/testapps/el7/software/staging/imkl/2022.2.1/compiler/2022.2.1/linux/compiler/lib/intel64_lin/libimf.so (0x00007f96dee8c000)
libirng.so => /opt/apps/testapps/el7/software/staging/imkl/2022.2.1/compiler/2022.2.1/linux/compiler/lib/intel64_lin/libirng.so (0x00007f96deb22000)
libcilkrts.so.5 => /opt/apps/testapps/el7/software/staging/intel-compilers/2022.2.1/compiler/2022.2.1/linux/compiler/lib/intel64_lin/libcilkrts.so.5 (0x00007f96de8e5000)
libintlc.so.5 => /opt/apps/testapps/el7/software/staging/imkl/2022.2.1/compiler/2022.2.1/linux/compiler/lib/intel64_lin/libintlc.so.5 (0x00007f96de873000)
libsvml.so => /opt/apps/testapps/el7/software/staging/imkl/2022.2.1/compiler/2022.2.1/linux/compiler/lib/intel64_lin/libsvml.so (0x00007f96dd262000)
/lib64/ld-linux-x86-64.so.2 (0x00007f96ecc7a000)
libirc.so => /opt/apps/testapps/el7/software/staging/imkl/2022.2.1/compiler/2022.2.1/linux/compiler/lib/intel64_lin/libirc.so (0x00007f96dd1f0000)
libifcoremt.so.5 => /opt/apps/testapps/el7/software/staging/intel-compilers/2022.2.1/compiler/2022.2.1/linux/compiler/lib/intel64_lin/libifcoremt.so.5 (0x00007f96dd081000)

mathieumorlighem

is this from within the job you submitted?

adamjhepburn

mathieumorlighem It is yeah, I added it on the line before the srun command

adamjhepburn

adamjhepburn yeah... I will keep digging tomorrow and let you know if I find anything! Thanks for your help

mathieumorlighem

then libpetsc.so is found....

mathieumorlighem

show us your queue just in case I see something obvious

adamjhepburn

#!/bin/bash
#SBATCH --job-name=STEADY_STATE_GlaDS_NoFluxGate-06-06-2024-16-37
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH --time=5
#SBATCH --mem-per-cpu=1gb
#SBATCH --output=STEADY_STATE_GlaDS_NoFluxGate-06-06-2024-16-37.outlog
#SBATCH --error=STEADY_STATE_GlaDS_NoFluxGate-06-06-2024-16-37.errlog

export ISSM_DIR="/users/gg4ahe/issm/bin/../"
source /users/gg4ahe/issm/etc/environment.sh

cd /mnt/parscratch/users/gg4ahe/SLIDE/STEADY_STATE_GlaDS_NoFluxGate-06-06-2024-16-37

ldd /users/gg4ahe/issm/bin/issm.exe
srun -n 1 /users/gg4ahe/issm/bin/issm.exe TransientSolution /mnt/parscratch/users/gg4ahe/SLIDE/STEADY_STATE_GlaDS_NoFluxGate-06-06-2024-16-37 STEADY_STATE_GlaDS_NoFluxGate-06-06-2024-16-37

mathieumorlighem

that should work 🙁

adamjhepburn

mathieumorlighem hmm okay... I will try a new install tomorrow with a different set of compilers and let you know how I get on!

adamjhepburn

mathieumorlighem

There must have been some sort of incompatibility with open-mpi along the line somewhere. Rerunning everything with intel compilers, letting PETSc install mpich, and forcing m1qn3 to use mpif90 seems to have done the trick. Thanks for your help!

Adam