Using the filetransfer QoS for Data Transfers

Created by Bent Petersen, Modified on Mon, 3 Mar at 3:41 PM by Bent Petersen

The filetransfer QoS is a higher-priority Slurm QoS designed only for data migration (e.g., moving files from one storage to a new location, SFTP etc.). It ensures faster scheduling compared to regular compute jobs while enforcing:

• 1 CPU

• 20 GB memory

• Maximum 4 days of runtime

This keeps data transfers fast and efficient without impacting compute workloads.

Priority and Scheduling

The filetransfer QoS is configured with a higher priority than standard compute partitions. This means:

✅ Filetransfer jobs will start faster than normal jobs in other queues, assuming resources are available.

✅ FairShare does not delay migration—even if your compute jobs are deprioritized.

✅ Only jobs with rsyncqos can run here, preventing misuse.

Requirements

1. Account & QoS Access

• You must belong to the mjolnir account.

• Your account must include the filetransfer QoS.

• Create a ticket if you see “Invalid qos specification.” after submitting a job

2. No Compute Jobs Allowed

• The filetransfer QoS is strictly for rsync or other related file-copy commands, such as SFTP etc.

• Compute jobs here could be killed, and the user may lose access to the filetransfer QoS.

Submitting a filetransfer QoS Job

sbatch --partition=filetransfer \
       --qos=filetransfer \
       --cpus-per-task=1 \
       --mem=20G \
       --time=1-00:00:00 \
       --wrap="rsync -av /path/to/source/ /path/to/destination/"

sbatch --partition=filetransfer \
       --qos=filetransfer \
       --cpus-per-task=1 \
       --mem=20G \
       --time=1-00:00:00 \
       --wrap="rsync -av --remove-source-files /path/to/source/ /path/to/destination/"

Explanation of Flags:

• --partition=filetransfer → Submits the job to the filetransfer partition.

• --qos=filetransfer → Enforces 1 CPU / 20GB memory limit.

• --cpus-per-task=1 → Must be 1 CPU.

• --mem=20G → Matches the allowed memory.

• --time=1-00:00:00 → Maximum runtime (adjust up to 4 days if needed).

• --wrap="rsync -av --remove-source-files /src /dest" → The actual rsync command.

Note: --remove-source-files deletes files from the source only after a successful copy. It will leave empty directories; you must delete them manually.

Verifying Your Job

1. Check the Queue

squeue -u $USER --partition=filetransfer

If your job is in PD (pending) status, check its reason:

scontrol show job <JOB_ID>

Common reasons include Resources or (QOSMaxCpuPerJobLimit) if you accidentally requested more than 1 CPU or 20GB memory.