Memory for implicit analyses

Home » Knowledge » Implicit » Memory for implicit analyses

In this section, we will distinguish between

  • the initialization phase (keyword reading, mpp decomposition etc.), and
  • the solution phase

of a typical LS-DYNA solution. During the initialization phase, LS-DYNA reads through all the keywords of the input file, which is then stored in RAM. For this phase, the upper limit of used memory is given by the command line options “memory=” and “memory2=”. If too little memory is given, LS-DYNA will issue the error OTH+21,

 *** Error 70021 (OTH+21) (processor # 0)
     Memory is set XXX words short
     ...

Instructions how to set “memory=” on the command line are also printed, and normally following these should resolve the problem. Note that the setting of “memory=” is an upper limit on the memory used – LS-DYNA will not allocate more memory than required for the initialization.

Once the solution phase starts, the major part of the memory allocation for implicit analyses is dynamic (since R11 of LS-DYNA). This means that LS-DYNA will allocate as much RAM memory as is required for the implicit solution. The command line options “memory=” and “memory2=” (or the options on *KEYWORD) will have no major effect on the amount of RAM used. It is possible to limit the amount of memory by the variables ABSMEM (absolute amount) and RDCMEM (relative amount) of the keyword *CONTROL_IMPLICIT_SOLVER.

When the analysis starts, the implicit solver of the Ansys LS-DYNA software will print an information message in the d3hsp file, for example,

================================================
==           IMPLICIT USAGE ALERT             ==
================================================
== Memory Management for Implicit has changed ==
== after R10. Please use:                     ==
==       memory= 5M memory2= 2M               ==
================================================

This is only for information. It is not an error message. Normally, no user action is required due to this message.

During a simulation, the amount of memory used will typically go up and down, as LS-DYNA allocates and releases memory as required for the different solution phases. If the exact amount of memory used by LS-DYNA is of interest, system monitoring tools (for example free in Linux) should be used. If a slurm queueing system is used, the command sstat (for a running job) and sacct (for a completed job) can be useful to retrieve estimated maximum memory used via the MaxRss parameter. LS-DYNA will also report an estimate of the available memory in the binary version of the glstat file, under the results label Mem_avl_GB_<hostname>.

If the available RAM memory is not sufficient for storing all required data for the matrix factorizations, LS-DYNA will try to use disk files for storage. This is often referred to as an “out-of-core” solution. The warning IMP+120 will be issued,

*** Warning 60120 (IMP+120)
    The stiffness matrix for this job is being factorized
    in out-of-core mode (using disk files), which may severely
    decrease performance.
    ...

If the print flag LPRINT of *CONTROL_IMPLICIT_SOLVER is > 1, the multifrontal linear solver will print a memory report in the mes0* file(s), typically

 symbolic storage 1 = 0.36
 in-core numerical storage 1 = 20.15
 out-of-core num. storage 1 = 5.79
 available numeric storg = 41.29 Mw
 symbolic storage 2 = 0.36
 in-core numerical storage 2 = 22.77
 out-of-core num. storage 2 = 4.75
 factor and solve should be in-core

if an “in-core” solution is possible (using only RAM storage). Even for an “out-of-core” solution, some RAM is required for the solution scheme. If the amount of RAM available is less than what is required for an “out-of-core” solution, LS-DYNA will stop, typically with a message like

*** Fatal Error - Implicit using F95 memory allocation
    could not allocate minimum amount of memory for 
    this factorization on at least one processor.
    Maximum of the minimum amount of memory per
    processor is ... Mwords.
    See d3hsp and mesxxxx files for additional information.

How to reduce the amount of RAM required

Beyond the obvious option to simplify the model by reducing the number of nodes and elements, the Ansys LS-DYNA software offers some interesting options of reducing the amount of RAM required for an implicit solution.

Using an iterative linear solver will in general save RAM memory. This can be activate by the variable LSOLVR (for example LSOLVR=22) on the keyword *CONTROL_IMPLICIT_SOLVER.

Activating the nonsymmetric linear solver by setting the variable LCPACK = 3 on *CONTROL_IMPLICIT_SOLVER will (roughly) double the RAM requirement, so if this is active but not strictly required for convergence, using the default symmetrical solver (LCPACK = 2) can save a lot of memory.

From R14, the parallelized version of METIS, ParMETIS, is available for the symbolic factorization. This is beneficial for scaling and speedup using MPP, but can also reduce the amount of RAM required. It is activated by setting ORDER=3 on the keyword *CONTROL_IMPLICIT_ORDERING.

From R15, more tools for potentially saving RAM are available via the keyword *CONTROL_IMPLICIT_SOLVER. Setting ISINGLE = 1 will activate use of mixed precision in the storage of the matrix factor. This may save 15 – 50 % of RAM. Memory balance between processes can be improved by setting MEMLVL = 1. Different options Block Low Rank (BLR) approximations are available via the variable IBLROPT.

The above applies for cases where the traditional FEM is used. There may be exceptions involving the mesh-free methods of LS-DYNA (SPH, SPG, etc.) , or method where a mesh is generated by the solver (ICFD, adaptivity, S-ALE etc.).

More details on this to follow.