How can I configure the Data Transfer Service container to use a specific network share or host directory for its project file staging area instead of using its default location?

The Data Transfer Service (DTS) relies on its own configuration file to define where it stages incoming project files before they are processed or moved to their final destination. If this is not explicitly configured, DTS may use a default location within its container or a standard Docker volume, which might not be the desired network share or persistent storage location.

To specify a custom staging directory (like a path on a mounted network share or a dedicated local file system), you must edit the DTS configuration file, typically located at config/dts/config_local.json relative to your main HPS deployment configuration directory.

Within this JSON file, locate the "storage" array. You must ensure that there is a storage definition where the "directory" key points explicitly to your desired staging path on the host system.

Here is an example configuration snippet for the storage section:

"storage": [
      {
          "buf_size": 10485760,
          "directory": "/path/to/staging", // <-- Set this to your actual staging path on the host
          "enabled": true,
          "name": "my_local", // This name might vary in your configuration
          "priority": 10,
          "type": "local",     
          "usage": "persistent" 
      }
      // ... potentially other storage definitions in the array ...
  ],

The key modification is setting the "directory" value correctly.


Important:  The path "/path/to/staging" used in the example above is fictional and for illustrative purposes only. You must replace this with the actual, valid absolute path on your host system which corresponds to your intended staging area (for example, /mnt/nfs/hpc_staging_area or /local_ssd/dts_staging). Ensure that this directory exists on the host and that the user/group running the DTS container process has the necessary read and write permissions for this path before starting the service.

After saving the changes to config/dts/config_local.json, you must restart the DTS container for the new configuration to be loaded and used. For example, use docker compose restart data-transfer or bring the stack down and up.