📍 Magus02 Knowledge Base → HPC User → Batch Scheduler
This page explains how to write and submit jobs to the Slurm batch scheduler on Magus02. For the theory behind scheduling, see Scheduling Basics.
Overview
All compute work on Magus02 must go through the Slurm batch scheduler. You write a jobscript — a shell script that declares resource requirements and the commands to execute — and submit it with sbatch. Slurm queues the job, waits for resources, and runs it on compute nodes.
⚠️ Common pitfalls to avoid:
• Requesting more memory than available → job stays in queue forever
• Requesting less time than needed → job is killed before it finishes
• Forgetting to load modules in jobscript → job fails silently
• Running heavy computations on login nodes → impacts all users
Essential Slurm Commands
| Command | Description |
|---|---|
sbatch jobscript.sh |
Submit a jobscript to the queue |
squeue |
List all jobs currently in the queue |
squeue -u $USER |
List only your jobs |
squeue -p medium |
List jobs in a specific partition |
scancel JOB_ID |
Cancel a specific job |
scancel -u $USER |
Cancel all your jobs |
scontrol show job JOB_ID |
Show detailed info about a job |
sinfo |
Show partition and node status |
sinfo -p medium |
Show status of a specific partition |
sacct -j JOB_ID |
Show accounting info for a completed job |
Common #SBATCH Directives
| Directive | Description |
|---|---|
#SBATCH --job-name=myjob |
Set a name for your job (visible in squeue) |
#SBATCH --partition=medium |
Choose the queue/partition |
#SBATCH --nodes=2 |
Number of compute nodes |
#SBATCH --ntasks=128 |
Total number of MPI tasks (processes) |
#SBATCH --ntasks-per-node=64 |
MPI tasks per node |
#SBATCH --cpus-per-task=2 |
CPU cores per task (for hybrid OpenMP+MPI) |
#SBATCH --mem=256G |
Total RAM per node |
#SBATCH --mem-per-cpu=4G |
RAM per CPU core |
#SBATCH --time=2-00:00:00 |
Wall time limit (D-HH:MM:SS) |
#SBATCH --output=slurm-%j.out |
STDOUT file (%j = job ID) |
#SBATCH --error=slurm-%j.err |
STDERR file |
#SBATCH --mail-type=END,FAIL |
Send email when job ends or fails |
#SBATCH [email protected] |
Email address for notifications |
Job Types
1. Serial Job (single core)
For programs that use only one CPU core. Use the small partition.
#!/bin/bash #SBATCH --job-name=serial_test #SBATCH --partition=small #SBATCH --nodes=1 #SBATCH --ntasks=1 #SBATCH --mem=8G #SBATCH --time=02:00:00 #SBATCH --output=serial_%j.out #SBATCH --error=serial_%j.err module purge module load intel/oneapi ./my_serial_program input.dat
For programs parallelized with OpenMP — multiple threads on a single node.
#!/bin/bash #SBATCH --job-name=openmp_job #SBATCH --partition=medium #SBATCH --nodes=1 #SBATCH --ntasks=1 #SBATCH --cpus-per-task=64 #SBATCH --mem=256G #SBATCH --time=12:00:00 #SBATCH --output=omp_%j.out #SBATCH --error=omp_%j.err module purge module load intel/oneapi export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK export OMP_PLACES=cores export OMP_PROC_BIND=close ./my_openmp_program
3. Distributed Memory Job (MPI)
For programs parallelized with MPI — multiple processes across one or more nodes.
#!/bin/bash #SBATCH --job-name=mpi_job #SBATCH --partition=medium #SBATCH --nodes=4 #SBATCH --ntasks-per-node=64 #SBATCH --mem=256G #SBATCH --time=7-00:00:00 #SBATCH --output=mpi_%j.out #SBATCH --error=mpi_%j.err module purge module load intel/oneapi module load mpi/intelmpi mpirun -n $SLURM_NTASKS ./my_mpi_program input.dat
4. Hybrid Job (MPI + OpenMP)
For programs that use both MPI (across nodes) and OpenMP (within each node). Requires careful configuration of both thread counts and MPI binding.
#!/bin/bash #SBATCH --job-name=hybrid_job #SBATCH --partition=large #SBATCH --nodes=4 #SBATCH --ntasks-per-node=8 #SBATCH --cpus-per-task=16 #SBATCH --mem=256G #SBATCH --time=14-00:00:00 #SBATCH --output=hybrid_%j.out #SBATCH --error=hybrid_%j.err module purge module load intel/oneapi module load mpi/intelmpi export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK mpirun -n $SLURM_NTASKS ./my_hybrid_program
5. GPU Job
For CUDA or GPU-accelerated programs. Always use the GPU partition.
#!/bin/bash #SBATCH --job-name=gpu_job #SBATCH --partition=GPU #SBATCH --nodes=1 #SBATCH --ntasks=16 #SBATCH --gres=gpu:4 #SBATCH --mem=256G #SBATCH --time=7-00:00:00 #SBATCH --output=gpu_%j.out #SBATCH --error=gpu_%j.err module purge module load cuda module load intel/oneapi ./my_cuda_program
6. Array Job
Run the same program multiple times with different inputs or parameters, all submitted as one job.
#!/bin/bash
#SBATCH --job-name=array_job
#SBATCH --partition=small
#SBATCH --nodes=1
#SBATCH --ntasks=8
#SBATCH --mem=32G
#SBATCH --time=04:00:00
#SBATCH --array=1-10
#SBATCH --output=array_%A_%a.out
#SBATCH --error=array_%A_%a.err
# $SLURM_ARRAY_TASK_ID is set to the array index (1..10)
module purge
module load intel/oneapi
./my_program input_${SLURM_ARRAY_TASK_ID}.dat
💡 Array job tip: Use array jobs instead of submitting hundreds of individual sbatch calls. They are tracked as a single entity and are more scheduler-friendly. Each task sees a unique $SLURM_ARRAY_TASK_ID so you can differentiate inputs.
7. Interactive Session
For testing and debugging interactively on a compute node (not the login node):
$ srun --pty --partition=small --nodes=1 --ntasks=8 --mem=32G --time=01:00:00 /bin/bash
This gives you a shell on a compute node. Exit with exit when done.
Monitoring Jobs
# Check the status of your jobs: $ squeue -u $USER # Output columns explained: # JOBID PARTITION NAME USER ST TIME NODES NODELIST # 12345 medium myjob deepak R 0:23 2 node[01-02] # Status: R=Running, PD=Pending, CG=Completing, F=Failed # Get detailed info on a specific job: $ scontrol show job 12345 # Check completed job stats (CPU time, memory used): $ sacct -j 12345 --format=JobID,Elapsed,MaxRSS,State
References
↑ Back to Knowledge Base · Maintained by HPC Administration · wiki.snu.edu.in
