When executing a job, you can provide custom information from the client side that allows you to perform custom actions prior to the submission of a job to the cluster. Custom information that you define on the RSM client machine can be picked up by RSM and then passed to the cluster submit host or cluster node where the job is being executed.
Examples of custom information that can be provided to the cluster are:
The username of the submitter (which, for instance, provides the ability to monitor jobs submitted by a particular user for accounting purposes)
The license necessary to execute the job, which can be used to integrate with cluster resource management to check Ansys license availability before a job starts running
For more information on how to integrate licensing with cluster software, contact your cluster administrator or Ansys customer support.
As an example, we’ll pass the submitter’s username from the client to a PBS Pro cluster.
The following sections detail the steps for providing custom information for job submissions to clusters.
First, you must define the information on the RSM client machine by creating an environment variable. The environment variable must begin with the prefix RSM_CLIENT_ in order for RSM to detect it and pass the information from the client machine to the cluster submit host.
In the example below, we’ve defined the environment variable RSM_CLIENT_USERNAME. The name is arbitrary as long as it begins with the RSM_CLIENT_ prefix.
Once you’ve defined the environment variable on the RSM client machine, it will be passed along with other job files to the cluster. You can access this environment variable value from your custom cluster job scripts. In our example, we will add the client job user name as a new command line argument to PBS Pro qsub command defined in the commands file RSM uses for PBS Pro clusters, hpc_commands_PBS.xml (located in the [RSMInstall]\Config\xml directory).
In the code sample below, you can see that the environment variable is added to the qsub command. Note, also, that it is preceded by –A, which defines the account string associated with the job for the PBS Pro cluster.
<primaryCommand name="submit">
<application>
<app>qsub</app>
</application>
<arguments>
<arg>
<value>-q %RSM_HPC_QUEUE%</value>
<condition>
<env name="RSM_HPC_QUEUE">ANY_VALUE</env>
</condition>
</arg>
<arg>
<value>-A %RSM_CLIENT_USERNAME%</value>
<condition>
<env name="RSM_CLIENT_USERNAME">ANY_VALUE</env>
</condition>
</arg>
<arg>
<value>-l select=%RSM_HPC_CORES%:ncpus=1:mpiprocs=1</value>
<condition>
<env name="RSM_HPC_DISTRIBUTED">TRUE</env>
</condition>
</arg>
<arg>
<value>-l select=1:ncpus=%RSM_HPC_CORES%:mpiprocs=%RSM_HPC_CORES%</value>
<condition>
<env name="RSM_HPC_DISTRIBUTED">FALSE</env>
</condition>
</arg>
<arg>-N "%RSM_HPC_JOBNAME%" %RSM_HPC_NATIVEOPTIONS% -V -o "%RSM_HPC_STAGING%/%RSM_HPC_STDOUTFILE%" -e
"%RSM_HPC_STAGING%/%RSM_HPC_STDERRFILE%" "%RSM_HPC_STAGING%/%RSM_HPC_COMMAND%"</arg>
</arguments>
</primaryCommand>
To view a sample of this file before the addition of custom information, see Modifying the Cluster-Specific HPC Commands File.
To verify that the custom information has been successfully passed from the RSM client to the cluster, run a job that will call the script you’ve customized. The environment variable should show up in the Reading environment variables… section of the RSM job log.
Reading environment variables... RSM_CLIENT_USERNAME = myname
Since we added the environment variable to the qsub command in the PBS Pro commands file, it will also show up in the area of the job log indicating that the qsub command has been run.
qsub -q %RSM_HPC_QUEUE% -A %RSM_CLIENT_USERNAME% -1 select=1:ncpus=%RSM_HPC_CORES%:mpiprocs=%RSM_HPC_CORES% ... qsub -q WB_pbsnat -A myname -1 select=1:ncpus=1:mpiprocs=1 ...