Distributed Analysis Configuration - Job Distribution Tab

On the Analysis Configuration window, the Job Distribution tab allows you to manually enable specific types of job distribution and enable multi-level solves.

The Job Distribution tab is disabled if you selected Use Automatic Settings.

The job distribution list allows you to specify which job distribution types to allow for the current analysis configuration. Use the check boxes enable and disable distribution types. At solve time, HFSS 3D Layout automatically selects the best distribution type from the enabled types.

Enabling a distribution type does not mean it will be used. It must be also allowed by the solve setup. If you enable a distribution type for a given setup, and distribution is allowed, the preview window updates to describe the distributions. Note that enabled distribution types apply to all setups of the given design type, so it is possible for different setups in a design to be solved using different distribution types.

The concurrent initial mesh generation workflow with Distributed Mesh Assembly relies on the MPI based distribution technology inside the MeshAssemblyManager. The decision whether to launch sequential or parallel mesh generation is based on the combination of the number of individual meshes present, the HPC setting, and the number of tasks available.

• If there’s only one single mesh or if curvilinear contact exists: then normal initial mesh workflow with single G3dmesher will be launched.
• If there are more than one individual mesh and no curvilinear contact exists:
  • If Use Automatic Setting is not selected, under the Job Distribution tab, if “Mesh Assembly” is not enabled, then meshes will be generated sequentially. If “Mesh Assembly” is enabled and user assigns more than 1 tasks, then parallel mesh generation will be launched.
  • If Use Automatic Setting is selected and user assigns more than 1 core, then parallel mesh generation will be launched.
  • If Use Automatic Setting, the number of MPI tasks will be the number of cores user assigned or the number of geometries to be meshed, whichever is smaller. If there are more geometries than cores, the geometries will be dynamically assigned to tasks. If there are more cores than geometries, the remaining available cores will provide some of the tasks with multi-thread capabilities.

All the detailed progress information from the Mesher is suppressed and the progress will report the number of meshes being finished. All the mesh profiles will be available under profile report and the mesh feedback will also be available under mesh feedback tool.

Enabled distribution types are listed when you select a configuration in the HPC and Analysis Options window.

When you run a simulation, the Messages window describes distributions.

When you view the Solution Profile, distributions for unitCell show as parallel Volume Tasks.

Distribution Levels

For products and designs that support two-level distribution, you can select either single or two-level distribution.

If you select single level, one distribution type will be applied at each stage of the solution process. If multiple types are available, the higher-level solution will generally be distributed. All machine tasks will be used by the single-level distribution.

Single-level Distributions

In a single-level distribution, one distribution type is applied at each stage of the solution process. Common stages include LastAdaptive, Sweep, and Parametric. All machine tasks will be used by the single-level distribution.

Supported distribution types include:

• Optimetrics Variations
• Frequencies
• Mesh Assembly
• Iterative Solver Excitations(for Technical details, see Multiprocessing and the Iterative Solver)
• Direct Solver Memory(for Technical details, see Distributed Memory Solutions with HFSS)
• Q3D Distributed Nets/Sources
• Q3D Matrix Solver

Solver distributions require MPI. See: Setting HPC and Analysis Options.

Parallel distribution types (such as Optimetrics Variations, Frequencies, and Excitations) do not require distribution. If these types are not able to distribute, the simulation can run sequentially.

Memory distribution types (such as Direct Solver Memory and Domain Solver) are set to require distribution. If these types are enabled, the software will assume that distribution is necessary to extend the simulation scale or add fundamental solution capabilities.

When multiple distribution types are available, the higher-level solution will generally be distributed. For example, when both Optimetrics Variations and Iterative Solver Excitations are enabled, Optimetrics Variations will be distributed. When both Optimetrics and Mesh Assembly, are selected Optimetrics are distributed. Domain Solver, Direct Solver Memory, Q3D Matrix Solver, and Q3D Distributed Net/Sources are exceptions because they are required; even though they are lower level, these types are distributed instead of parallel distribution types.

Two-Level Distributions

Selecting Enable two-level enables the Distributed solutions at first level box.

In a two-level distribution, the first level distributes the specified number of solutions. Each solution will then use a subset of machine tasks to distribute the second level. A solver distribution type must be available for the second level; otherwise, single-level distribution will be applied.

For two-level distribution, the total number of tasks must be greater than or equal to the number of tasks for level 1.

The following are examples of two-level distributions:

• A parametric setup distributing Optimetrics variations as level 1 and iterative solver excitations as level 2.
• A parametric setup for a non-transient problem distributing optimetrics variations as level one and frequencies as level 2.
• A frequency sweep distributing frequencies as level 1 and direct solver memory as level 2.
• A parametric setup for a transient network problem distributing optimetrics variations as level one and transient excitations as level 2.

The Analysis Configuration window displays a preview of the job distribution, if allowed for that configuration.

For more information, see: Two-Level Distribution Guidelines.

Activating the Q3D Distribution List Options Beta Feature

Each of the new Q3D job distribution types improve performance on large designs by parallelizing different parts of the solution. Distributed Nets/Sources targets problems with hundreds of nets or sources by splitting MPI processes into groups. Each group holds the full matrix, while different nets are distributed across the groups. This improves scaling when solve time is significantly greater than setup time. The solver automatically chooses the optimal number of groups based on memory profiling and the solve/setup ratio. Matrix Solver partitions the matrix itself across MPI processes so that very large problems can be solved when a single machine’s memory is insufficient. By redistributing the mesh so that each MPI task handles only a partial mesh, and by parallelizing tasks, these new distribution types allows the simulation to handle designs containing 10 million+ mesh elements.

Follow these steps to add Q3D Distributed Nets/Sources and Q3D Matrix Solver to the distribution list.

1. From the Tools menu, select Options > General Options to open the Options window

   

2. Expand General > Desktop Configuration and click Beta Options to open the Beta Options window

3. Scroll down to the HFSS 3D Layout section and check the box adjacent to Enable Q3D ACRL Distributed Memory.

   

4. Click OK to close the Beta Options window.

5. Click OK to close the Options window and open a prompt to restart the software. Users may be asked to save their work.

   

6. Click Yes to restart immediately.

7. After restarting the software, Q3D Distributed Nets/Sources and Q3D Matrix Solver will be visible in the distribution types list.

   

8. Select the appropriate option.

The procedure is complete.