Memory requirements within Mechanical APDL are driven primarily by the requirement of solving large systems of equations. Details of solver memory requirements are discussed in a later section. Additional memory demands can come from meshing large components and from other pre- and postprocessing steps which require large data sets to be memory resident (also discussed in a later section).
Another often-overlooked component of memory usage comes from a hidden benefit when large amounts of physical memory are available on a system. This hidden benefit is the ability of operating systems to use available physical memory to cache file I/O. Maximum system performance for simulations occurs when there is sufficient physical memory to comfortably run the equation solver while also caching the large files generated during the simulation.
Mechanical APDL memory is divided into two blocks: the database space that holds the current model data and the scratch space that is used for temporary calculation space (used, for example, for forming graphics images and by the solvers). The database space is specified by the -db command line option. The initial allocation of total workspace is specified by the -m command line option. The scratch space is the total workspace minus the database space. Understanding how scratch space is used (as we will see in later chapters) can be an important component of achieving optimal performance with some of the solver options.
In general, specifying a total workspace (-m) or database memory (-db) setting at startup is no longer necessary. Both the scratch space and database space (64-bit systems only) grow dynamically, provided the memory is available when the memory manager tries to allocate additional memory from the operating system. If the database space is unavailable (or forced not to grow dynamically via a negative -db value), the program automatically uses a disk file (.page) to spill the database to disk. See Memory Management and Configuration in the Basic Analysis Guide for additional details on memory management.
Memory considerations for shared memory parallel (SMP) processing are essentially the same as for running Mechanical APDL on a single processor. All shared memory processors access the same user memory, so there is no major difference in memory demands for SMP mode.
DMP memory usage requires more explanation. When running DMP using n cores,
n
distributed processes are started. The first of these processes is often
referred to as the master or rank-0 process, while the remaining
n-1 processes are often referred to as the worker
processes.
The master MPI process always does more work than the remaining processes, and therefore always requires more memory than the worker processes. The master process reads the entire input file and does all of the pre- and postprocessing, as well as the initial model decomposition required for DMP. In addition, while much of the memory used for the SOLVE command scales across the processes, some additional solver memory requirements are unique to the master process.
When running on a cluster, the memory available on the compute node containing the master process (or head compute node) will typically determine the maximum size of problem that can be solved. Generally, the compute node that contains the master process should have twice as much memory as all other machines used for the run. As an example, a cluster of 8 compute nodes with 4 GB each cannot solve as large a problem as an SMP machine that has 32 GB of memory, even though the total memory in each system is the same. Upgrading the head compute node to 8 GB, however, should allow the cluster to solve a similar-sized problem as the 32 GB SMP system.