Welcome to HPC Workshop 4 - Advanced Slurm — an essential part of your journey toward mastering High Performance Computing (HPC) with HSUper!

This training is part of the reimagined HSUper Workshops series, originally offered as a six-part on-site program and now presented as flexible, self-paced digital trainings.

Learning Goals

• Job Priorities and Updates: You will learn about what various Slurm messages on HSUper mean, what might be hindering your job from running, and how to update your job to work around the limitations set by HSUper. You will also learn about job priorities, how to check the priorities of your job, and tips on getting your jobs running faster.

• Job Dependencies: Job dependencies can be used to set up orders and dependencies for job execution. You will learn about Slurm’s dependency system, some common uses, and some advanced tips.

• Job Arrays: Job Arrays are a mechanic used to submit a very large number of jobs (up to 1,000) with similar configurations. The main use case is to have many small and quick jobs that can be submitted and handled as one batch - an array. The advantages of this approach include slurm taking account of their similarity and queueing them faster (less waiting time), and the possibility of managing them directly as a batch.

• Checkpointing: A program or simulation can save intermediate states to pick up from, usually called checkpoints. You will learn some advantages of and use cases for checkpoints and which tools Slurm provides for this.

• Preemption: On HSUper, preemption is used to discard the limit of maximum concurrent running jobs per user. You will learn how to use preemption on HSUper and what the advantages and disadvantages are.

• Job Arguments: In an optional task, you will learn how to pass arguments calling your job with sbatch to your job script.

Level of Difficulty

This workshop is tailored for HSUper users at the Advanced-Level who are already familiar with the basics HPC, can navigate around HSUper and are already working on projects using methods, software and hardware related to HPC.

Requirements

While this course is free to use for everyone, you will need access to HSUper and access to the campus network (VPN possible), to actually work through it.

• Access to HSUper

You should already have acquired the skills from the previous workshops:

• HPC Workshop 1 - A short introduction to HSUper and the Linux Terminal
• HPC Workshop 2 - Data Transfer and System Resources
• HPC Workshop 3 - Module Systems and Slurm Jobs