This issue arises because the web frontend components, specifically the Job Management Service (JMS) container, need to temporarily cache the uploaded job data locally before moving it to the final job staging area (which is typically a network share, often on a Parallel File System - pFS). This caching happens within the container's file system, often mapping back to the host's /var directory by default.
If the total size of the data being cached exceeds the available space in the host's partition backing the container's cache directories (like /var and potentially /tmp), the partition will fill up, potentially causing errors or system instability.
Solutions: There are two main solutions to prevent this. Choose the solution that best fits your infrastructure and deployment strategy.
Pre-deployment Host Partition Sizing: Before deploying the HPS core services, ensure that the host machine's /var partition (or whichever partition will implicitly back the container's temporary storage) is sized large enough to handle the anticipated peak load of cached job data from concurrent uploads.
Container Volume Mapping Adjustment: Before building and deploying the HPS core services using Docker Compose, you can modify the docker-compose.yaml file. Specifically, locate the service definition for the jms container and adjust its volume mappings. Map the container's internal /var and /tmp directories to volumes that reside on a different, larger local filesystem or directly onto a suitable network share that has sufficient space. This redirects the temporary caching away from the potentially limited default host partition.
Below is a conceptual example showing how the
volumessection for thejmsservice might look in your docker-compose.yaml file after modification. This example assumes that you have a large file system or network share mounted at/mnt/hpc_staging_cacheon the host machine.services: jms: # ... other jms configuration lines (image, ports, environment, etc.) ... volumes: # Map container /var to a dedicated directory on the host's large storage - /mnt/hpc_staging_cache/jms/var:/var # Map container /tmp to a dedicated directory on the host's large storage - /mnt/hpc_staging_cache/jms/tmp:/tmp # ... potentially other existing volume mappings for jms ... # ... other services defined in your docker-compose.yaml ... ```Important: The path
/mnt/hpc_staging_cache/used in this example is fictional and for illustrative purposes only. You must replace/mnt/hpc_staging_cache/jms/varand/mnt/hpc_staging_cache/jms/tmpwith the actual, valid paths on your host system that point to your chosen storage location (which needs sufficient space and appropriate permissions for the container user) before applying this change and deploying the containers. Ensure that the target directories exist on the host system.