5.5.2. Distributed Parallel Using a Queuing System

To use a queuing system to run your job is very straightforward since all you need to do is call run_mpi.sh within your job submission file or command. Ansys Forte supports both the Altair Grid Engine (formerly UGE) and the Simple Linux Utility for Resource Management (SLURM) https://slurm.schedmd.com/documentation.html. queueing systems.

Below are examples of Forte job scripts for both of these systems. These examples should be sufficient to get you started on such systems, though you may need to contact your IT department to find out which resources (parallel environments or partitions, etc.) are available to you.


Note:  It may be possible to run Forte using other queueing systems, such as, for example, PBS, but we have not tested them and therefore do not support other systems.


5.5.2.1. UGE Cluster Example

Below is an example of a job submission script to submit a Forte job to a Linux UGE (Altair Grid Engine) cluster using 28 cores on the "pe_mpi" parallel environment.

#!/bin/bash 
#$ -N MyForteRun 
#$ -pe pe_mpi 28 
#$ -q ugequeue 
#$ -o $HOME/SGE_Logs/$JOB_ID 
#$ -V 
#$ -j y 
#$ -m abes 
# CD To RUN DIRECTORY (assumes you submit job from this directory) 
cd $SGE_O_WORKDIR 
# Call the Forte script to run Forte in parallel 
/bin/sh ./run_mpi.sh 

5.5.2.2. Slurm Cluster Example

Below is an example of a job submission script to submit a Forte job to a Slurm cluster, specifying 112 cores on "partition01" where each node uses 28 cores.

#!/bin/bash 
#SBATCH -J 
MyForte Run
#SBATCH -p partition01 
#SBATCH --wait-all-nodes=1
#SBATCH --mail-user="any.user@domain.com" 
#SBATCH --mail-type=all
#SBATCH --exclusive 
#SBATCH -n 112
#SBATCH --ntasks-per-node 28 

cd $SLURM_SUBMIT_DIR
/bin/sh ./run_mpi.sh