`shelving` variables fails using `savevars` function

pf4d

Hi there,

I can successfully save my model using the savevars function for small problems, but if I have a large problem---or a small problem that has been solved over many time steps---the savevars function simply fails. I have plenty of RAM, and never never get close to using all of it. This behavior is really inconvenient as I lose the entire simulation; for example, after a simulation I get this final message:

write lock file:

   FemModel initialization elapsed time:   0.636059
   Core solution elapsed time:             37978

   Total elapsed time: 10 hrs 32 min 58 sec
closing PETSc
closing MPI
loading results from cluster
Shelving variables to new file './results/BP/shelf_bc_subelement_slip_fric_1e4_iceFront_dx_5000.md'.
Variable 'md' not shelved.

I can "fix" the problem by increasing the parameter md.settings.output_frequency, but I see no particular reason this must be so.

Digging around, I see that the data are saved each timestep, but is saved in the $ISSM_ROOT/execution directory as an outbin file. Is there a way to change this behavior so that it outputs these data into the execution directory?

in-woo

Hi, Evan.

If you want to save the model, you would use 'savemodel' function in matlab.

Maybe you have to test the example of ISSM model at
"https://issm.jpl.nasa.gov/documentation/"

Also, you can control the time step of model and saving model frequency as following code.

md.timestepping.start_time = 2007;  % Initial time of model simulation
md.timestepping.time_step  = .01; % If 'md.timestepping.time_adapth = 0', you have to set the time step of transient simulation.
md.timestepping.time_adapt = 0;  % you can use automatic time step.
md.timestepping.final_time = md.timestepping.start_time+100; % End time of model simulation.
md.settings.output_frequency = 100;  % As following time step, you can get the model results every 1 years. 
                                                         % For example, result of time step is ;

Best Inwoo.

pf4d

Hi Inwoo,

I do not have Matlab, I only have Python. Since originally writing this question, I've determined that I can avoid breaking the savevars method if I only save a few variables. To do this, I create a dictionary, such as:

var_dict  = {'md.mask.groundedice_levelset' : md.mask.groundedice_levelset,
             'md.mask.ice_levelset'         : md.mask.ice_levelset,
             'md.geometry.surface'          : md.geometry.surface,
             'md.geometry.base'             : md.geometry.base,
             'md.geometry.thickness'        : md.geometry.thickness,
             'md.inversion.vx_obs'          : md.inversion.vx_obs,
             'md.inversion.vy_obs'          : md.inversion.vy_obs,
             'md.inversion.vel_obs'         : md.inversion.vel_obs,
             'md.friction_coefficient'      : md.friction.coefficient}
im.savevars(out_dir + 'issm_nio.shelve', var_dict)

instead of saving the whole model like

im.savevars(out_dir + 'issm_refined_mesh.md', 'md', md)

which crashes.

Regarding my second question: I know how to set the timestep parameters, etc... Rather I was inquiring about why execution data are saved by default within the installation directory rather than the execution directory.

in-woo

Hi, Evan.

My poor understanding gives the wrong answer. Sorry.

It's so sad that I can not help you.

Best Inwoo

mathieumorlighem

Hi guys,

This is one of the reasons why python is not recommended. You can take a look at src/m/contrib/defleurian/netCDF/export_netCDF.py if you want to save your model as netCDF instead of the default python format.

I hope this helps
Best
Mathieu

weiji14

Thanks for the hint to the export_netCDF function! I've managed to export the md model to a netCDF file, but the 'read_netCDF` script doesn't seem to work.

from read_netCDF import netCDFRead

netCDFRead(filename="md.nc")

is giving this error:

Opening md.nc for reading 
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-9-c017d4078f17> in <module>
----> 1 netCDFRead(filename="md.nc")

/home/jovyan/pyissm/trunk/src/m/contrib/defleurian/netCDF/read_netCDF.py in netCDFRead(filename)
     18         for children in walktree(NCData):
     19             for child in children:
---> 20                 class_dict[str(child)] = str(getattr(NCData.groups[str(child)], 'classtype') + '()')
     21 
     22         print(class_dict)

KeyError: 'variables'

I've also tried using savevars to save md to a .dat file and that works, but loadmodel doesn't work:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
/home/jovyan/miniconda3/envs/pyissm/lib/python3.8/shelve.py in __getitem__(self, key)
    110         try:
--> 111             value = self.cache[key]
    112         except KeyError:

KeyError: 'md'

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
<ipython-input-15-e41e41fdf466> in <module>
----> 1 loadmodel("md.dat")

/home/jovyan/pyissm/trunk/src/m/io/loadmodel.py in loadmodel(path)
     33     #       try:
     34     #recover model on file and name it md
---> 35     struc = loadvars(path)
     36     name = [key for key in list(struc.keys())]
     37     if len(name) > 1:

/home/jovyan/pyissm/trunk/src/m/io/loadvars.py in loadvars(*args)
     74         else:
     75             for name in list(my_shelf.keys()):
---> 76                 nvdict[name] = my_shelf[name]
     77                 print(("Variable '%s' loaded." % name))
     78         my_shelf.close()

/home/jovyan/miniconda3/envs/pyissm/lib/python3.8/shelve.py in __getitem__(self, key)
    112         except KeyError:
    113             f = BytesIO(self.dict[key.encode(self.keyencoding)])
--> 114             value = Unpickler(f).load()
    115             if self.writeback:
    116                 self.cache[key] = value

/home/jovyan/pyissm/trunk/src/m/qmu/helpers.py in __getattr__(self, attr)
    147         try:
    148             # check if in keys, then access
--> 149             _k = object.__getattribute__(self, '_k')
    150             _v = object.__getattribute__(self, '_v')
    151             pos = _k.index(attr)

AttributeError: 'OrderedStruct' object has no attribute '_k'

Just wondering if this is a known issue, and if there is a way to load a saved model up somehow?

weiji14

Right, I figured it out. The 'trick' is to save it to a NetCDF file, and use loadmodel to load it...

export_netCDF(md=md, filename="model.nc")  # save model
md = loadmodel(path="model.nc")  # load model