5.5.1. Distributed Parallel on Linux Using a Hostfile

Forte supports all the methods to control process placement as supported and as documented by Intel for the version of MPI you are using (see Appendix B: Forte MPI Support and Known Issues). Below are examples of each process control method for a Linux cluster. For full details of the options that Intel MPI provides to control process placement, see: https://www.intel.com/content/www/us/en/developer/articles/technical/controlling-process-placement-with-the-intel-mpi-library.html.

Each of these methods requires you to add additional arguments to the mpirun command in the run_env.sh file. There are two ways to do this:

  1. Modify the Linux run settings options in the Forte User Interface before preparing the project. (see Windows Settings Panel and Linux Settings Panel).

  2. Modifying via the Forte Command Line Interface prepare command to include the hostfile as follows:

    Create a Forte Command Line Interface settings file, for example run_settings.txt. with the following content:

    mpi_command_l = <modified mpirun line as described in section (1), (2) or (3) below>

    For example:

    mpi_command_l = "mpirun -machine $HOME/machines.txt @FORTE"
END

    Then include this settings file as part of the Forte Command Line Interface prepare step:

    forte.sh CLI -project <MyProject.ftsim> --prepare -mpi_args <number of MPI processes> \
-import run_settings.txt

Ansys recommends you use one of these supported approaches instead of manually editing the run_mpi script directly yourself.

The following are examples of how you can modify the mpirun command to define your distributed parallel host resources:

(1) Specifying Hosts

Running 2 processes each on nodeA and nodeB:

mpirun -n 4 -ppn 2 -hosts nodeA,nodeB @FORTE

(2) Using a Machine File

Using a machine file to place on Linux:

  • 2 processes on node NodeA and

  • 4 processes on NodeB 

mpirun  -machine $HOME/machines.txt @FORTE

where the contents of $HOME/machines.txt is:

nodeA:2
nodeB:4

(3) Using Argument Sets

Running 3 processes on nodeA, 4 on nodeB, and 3 on nodeC, edit the Run Settings > Linux Settings > Parallel Execution Command as follows:

mpirun -host nodeA -n 3 @FORTE : -host nodeB -n 4 @FORTE : -host nodeC -n 3 @FORTE