Memory requirements within Mechanical APDL are driven primarily by the requirement of solving large systems of equations. Details of solver memory requirements are discussed in a later section. Additional memory demands can come from meshing large components and from other pre- and postprocessing steps which require large data sets to be memory resident (also discussed in a later section).
Another often-overlooked component of memory usage comes from a hidden benefit when large amounts of physical memory are available on a system. This hidden benefit is the ability of operating systems to use available physical memory to cache file I/O. Maximum system performance for simulations occurs when there is sufficient physical memory to comfortably run the equation solver while also caching the large files generated during the simulation.
Mechanical APDL memory is divided into two blocks: the database space that holds the current model data and the scratch space that is used for temporary calculation space (used, for example, for forming graphics images and by the solvers). The database space is specified by the -db command line option. The initial allocation of total workspace is specified by the -m command line option. The scratch space is the total workspace minus the database space. Understanding how scratch space is used (as we will see in later chapters) can be an important component of achieving optimal performance with some of the solver options.
In general, specifying a total workspace (-m) or database memory (-db) setting at startup is no longer necessary. Both the scratch space and database space (64-bit systems only) grow dynamically, provided the memory is available when the memory manager tries to allocate additional memory from the operating system. If the database space is unavailable (or forced not to grow dynamically via a negative -db value), the program automatically uses a disk file (.page) to spill the database to disk. See Memory Management and Configuration in the Basic Analysis Guide for additional details on memory management.
Memory considerations for shared memory parallel (SMP) processing are essentially the same as for running Mechanical APDL on a single processor. All shared memory processors access the same user memory, so there is no major difference in memory demands for SMP mode.
DMP memory usage requires more explanation. When running
DMP using n cores,
n
distributed processes are started. The first of these processes is
often referred to as the master or rank-0 process, while the remaining
n-1 processes are often referred to as the worker
processes.
The master MPI process always does more work than the remaining processes, and therefore always requires more memory than the worker processes. The master process reads the entire input file and does all of the pre- and postprocessing, as well as the initial model decomposition required for DMP. In addition, while much of the memory used for the SOLVE command scales across the processes, some additional solver memory requirements are unique to the master process.
When running on a cluster, the memory available on the compute node containing the master process (or head compute node) will typically determine the maximum size of problem that can be solved. Memory limitations on the head node can generally be resolved in two ways. The first is to place fewer processes on the head node. This provides more available memory to the master process. The second is to physically install more memory on node and designate it as the head node. Since most clusters are homogenous, this may not be practical. If it is, however, then the guidance is to install twice as much memory on the head node as the other compute nodes.