43.4. Starting Parallel Ansys Fluent on a Linux System

You can run Ansys Fluent on a Linux system using either the graphical user interface (for details, see Starting Parallel Ansys Fluent Using Fluent Launcher) or command line options (for details, see Starting Parallel Ansys Fluent on a Linux System Using Command Line Options and Setting Up Your Secure Shell Clients).

Important:  On some clusters without accelerated graphics, Fluent may not accept keyboard inputs. If you encounter this behavior, set the QT_XKB_CONFIG_ROOT environment variable equal to /usr/share/X11/xkb.

For additional information, see the following sections:

 

43.4.1. Starting Parallel Ansys Fluent on a Linux System Using Command Line Options

To start the parallel version of Ansys Fluent using command line options, you can use the following syntax in a command prompt window:

fluent version -t nprocs [-gpgpu= ngpgpus ] [-p interconnect ] [-mpi= mpi_type ] [-cnf= hosts ]

where

  • version must be replaced by the version of Ansys Fluent you want to run (2d, 3d, 2ddp, or 3ddp).

  • -t nprocs specifies the number of processes to use. When the -cnf option is present, the hosts argument is used to determine which machines to use for the parallel job. For example, if there are 10 machines listed in the hosts file and you want to run a job with 5 processes, set nprocs to 5 (that is, -t5) and Ansys Fluent will use the first 5 machines listed in the hosts file. Note that if the -gpgpu option is used, nprocs must be chosen such that the number of solver processes per machine is equal on all machines.

  • -p interconnect (optional) specifies the type of interconnect. The auto-select interconnect is used by default so that the best available interconnect is used if the option is not explicitly specified. See Table 43.5: Supported Interconnects for Linux Platforms (Per Platform), Table 43.6: Available MPIs for Linux Platforms, and Table 43.7: Supported MPIs for Linux Architectures (Per Interconnect) for more information.

  • -mpi= mpi_type (optional) specifies the type of MPI. If the option is not specified, the default MPI for the given interconnect will be used (the use of the default MPI is recommended). The available MPIs for Linux are shown in Table 43.6: Available MPIs for Linux Platforms.

  • -cnf= hosts (optional) specifies the hosts file, which contains a list of the machines on which you want to run the parallel job; if this option is not used, then the session will run on the local machine. If the hosts file is not located in the directory where you are typing the startup command, you must supply the full pathname to the file. You may also need to supply the full pathname in some scheduler and custom-scripted scenarios; you will know this is the case if Fluent prints the following error message:

    Error: file "hosts.txt" does not exist!

    You can use a plain text editor to create the hosts file. The only restriction on the filename is that there should be no spaces in it. For example, hosts.txt is an acceptable hosts file name, but my hosts.txt is not.

    Your hosts file (for example, hosts.txt) might contain the following entries:

     machine1 
 machine2

    If a machine in the network is a multiprocessor, you can list it more than once. For example, if machine1 has 2 CPUs, then, to take advantage of both CPUs, the hosts.txt file should list machine1 twice:

     machine1 
 machine1 
 machine2

    As an alternative to a hosts file, you can simply type the names of the machines in a list separated by commas, as shown in the following example: -cnf=machine1,machine1,machine2.

    By default, Fluent allocates ranks to machines in contiguous blocks, where the block sizes are as equal as possible. You can control the per machine block allocation size using the machine​ X : Y convention in the hosts specification, where Y is the process block count​ ​for ​machine​​ X. The process assignment ​will ​cycle through the machine list until all processes are allocated in specified ​blocks. ​A​ fully round-robin ​assignment of processes​ can be achieved by setting the machine block allocation sizes to 1 (for example, machine1:1, machine2:1, and so on).

  • -gpgpu= ngpgpus specifies the number of GPGPUs per machine to use in offload mode to accelerate the Algebraic Multigrid (AMG) solver and/or the Discrete Ordinates (DO) radiation model calculations. Note that when this option is used, the number of solver process per machine must be equal on all machines and ngpgpus must be chosen such that the number of solver processes per machine is an integer multiple of ngpgpus. That is, for nprocs solver processes running on M machines using ngpgpus GPGPUS per machine:

    See Using General Purpose Graphics Processing Units (GPGPUs) With the Algebraic Multigrid (AMG) Solver and/or Accelerating Discrete Ordinates (DO) Radiation Calculations for more information about using GPGPU acceleration.

For example, to use the InfiniBand interconnect, and to start the 3D solver with 4 compute nodes on the machines defined in the text file called fluent.hosts, you can enter the following in the command prompt:

fluent 3d -t4 -pinfiniband -cnf=fluent.hosts

Note that if the optional -cnf= hosts is specified, a compute node will be spawned on each machine listed in the file hosts.

During startup, Ansys Fluent will create very small text files in the /tmp area. If you would like to specify a different location for these files, set the following environment variable: export FL_TMPDIR= directory.

The supported interconnects for parallel Linux machines are listed below (Table 43.5: Supported Interconnects for Linux Platforms (Per Platform), Table 43.6: Available MPIs for Linux Platforms, and Table 43.7: Supported MPIs for Linux Architectures (Per Interconnect)), along with their associated communication libraries, the corresponding syntax,

Table 43.5: Supported Interconnects for Linux Platforms (Per Platform)

Platform Processor Architecture Interconnects/Systems*
Linux 64-bit lnamd64 ethernet, ethernet.efa**, infiniband

(*) Node processes on the same machine communicate by shared memory. Ansys Fluent lets the MPI autoselect the best interconnect available on the system. Users can specify an interconnect to override that selection. Ethernet is the fallback choice.

(**) ethernet.efa is only a valid setup in Amazon Web Service (AWS) instances with the EFA interface.

Table 43.6: Available MPIs for Linux Platforms

MPI Syntax (flag) Communication Library Notes
intel -mpi=intel Intel MPI General purpose for SMPs and clusters

(1)

openmpi -mpi=openmpi Open MPI Open source MPI-2 implementation. For both SMPs and clusters.

(2)


(1) By default, the Intel MPI may fail when mixing hardware for compute nodes. As a workaround, you can use the following environment setting:

I_MPI_PLATFORM none

(2) This version of Open MPI is dependent on the following versions (or higher) of software:

Table 43.7: Supported MPIs for Linux Architectures (Per Interconnect)

Architecture Ethernet Ethernet.EFA InfiniBand Omni-Path*
lnamd64 intel (default) and openmpi intel (default) and openmpi intel (default) and openmpi intel (default)

(*) For Intel Omni-Path architecture, Omni-Path software 10.2 or higher is recommended.

(Linux Only) To enable a job-scheduler-supported native remote node access mechanism, you can use the -scheduler_tight_coupling command line option. For details about the MPI / job scheduler combinations that are supported for this tight coupling, see Running Fluent Using a Load Manager.

 

43.4.2. Setting Up Your Secure Shell Clients

For cluster computing on Linux systems, most parallel versions of Ansys Fluent will require the user account set up such that you can connect to all nodes on the cluster (using the secure shell (ssh) client) without having to enter a password each time for each machine.

Provided that the server daemon (sshd) is running, this section briefly describes how you can configure your system in order to use Ansys Fluent for parallel computing.

 

43.4.2.1. Configuring the ssh Client

The secure shell client (ssh) is used widely. Depending on the specific protocol and the version deployed, configuration involves a few steps. SSH1 and SSH2 are two current protocols. OpenSSH is an open implementation of the SSH2 protocol and is backwards compatible with the SSH1 protocol. To add a client machine, with respect to user configuration, the following steps are involved:

  1. Generate a public-private key pair using ssh-keygen (or using a graphical user interface client). For example:

      % ssh-keygen -t dsa

    where it creates a Digital Signature Authority (DSA) type key pair.

  2. Place your public key on the remote host.

    • For SSH1, insert the contents of the client (~/.ssh/identity.pub) into the server (~/.ssh/authorized_keys).

    • For SSH2, insert the contents of the client (~/.ssh/id_dsa.pub) into the server (~/.ssh/authorized_keys2).

The client machine is now added to the access list and you are no longer required to type in a password each time. For additional information, consult your system administrator or refer to your system documentation.