Deepak: new

2026-04-20T06:41:21Z

new

New page

{{DISPLAYTITLE:Batch Scheduler — Magus02 HPC}}
<div style="background:#0d1b2a;border:1px solid #1e3a5a;border-radius:8px;padding:12px 18px;margin-bottom:20px;font-size:13px;font-family:monospace;color:#5ba3d9;">
📍 [[Magus02/Knowledge_Base|Magus02 Knowledge Base]] → HPC User → Batch Scheduler
</div>

This page explains how to write and submit jobs to the Slurm batch scheduler on Magus02. For the theory behind scheduling, see [[Magus02/Scheduling_Basics|Scheduling Basics]].

__TOC__

== Overview ==

All compute work on Magus02 must go through the '''Slurm''' batch scheduler. You write a '''jobscript''' — a shell script that declares resource requirements and the commands to execute — and submit it with <code>sbatch</code>. Slurm queues the job, waits for resources, and runs it on compute nodes.

<div style="background:#2a1a00;border-left:4px solid #f0c040;border-radius:6px;padding:14px 18px;margin:16px 0;font-size:14px;color:#d4b870;">
⚠️ <strong>Common pitfalls to avoid:</strong><br>
• Requesting more memory than available → job stays in queue forever<br>
• Requesting less time than needed → job is killed before it finishes<br>
• Forgetting to load modules in jobscript → job fails silently<br>
• Running heavy computations on login nodes → impacts all users
</div>

== Essential Slurm Commands ==

{| class="wikitable" style="width:100%;"
|-
! style="background:#0d1b2a;color:#94a8be;" | Command
! style="background:#0d1b2a;color:#94a8be;" | Description
|-
| <code>sbatch jobscript.sh</code> || Submit a jobscript to the queue
|-
| <code>squeue</code> || List all jobs currently in the queue
|-
| <code>squeue -u $USER</code> || List only your jobs
|-
| <code>squeue -p medium</code> || List jobs in a specific partition
|-
| <code>scancel JOB_ID</code> || Cancel a specific job
|-
| <code>scancel -u $USER</code> || Cancel all your jobs
|-
| <code>scontrol show job JOB_ID</code> || Show detailed info about a job
|-
| <code>sinfo</code> || Show partition and node status
|-
| <code>sinfo -p medium</code> || Show status of a specific partition
|-
| <code>sacct -j JOB_ID</code> || Show accounting info for a completed job
|}

== Common #SBATCH Directives ==

{| class="wikitable" style="width:100%;"
|-
! style="background:#0d1b2a;color:#94a8be;" | Directive
! style="background:#0d1b2a;color:#94a8be;" | Description
|-
| <code>#SBATCH --job-name=myjob</code> || Set a name for your job (visible in squeue)
|-
| <code>#SBATCH --partition=medium</code> || Choose the queue/partition
|-
| <code>#SBATCH --nodes=2</code> || Number of compute nodes
|-
| <code>#SBATCH --ntasks=128</code> || Total number of MPI tasks (processes)
|-
| <code>#SBATCH --ntasks-per-node=64</code> || MPI tasks per node
|-
| <code>#SBATCH --cpus-per-task=2</code> || CPU cores per task (for hybrid OpenMP+MPI)
|-
| <code>#SBATCH --mem=256G</code> || Total RAM per node
|-
| <code>#SBATCH --mem-per-cpu=4G</code> || RAM per CPU core
|-
| <code>#SBATCH --time=2-00:00:00</code> || Wall time limit (D-HH:MM:SS)
|-
| <code>#SBATCH --output=slurm-%j.out</code> || STDOUT file (%j = job ID)
|-
| <code>#SBATCH --error=slurm-%j.err</code> || STDERR file
|-
| <code>#SBATCH --mail-type=END,FAIL</code> || Send email when job ends or fails
|-
| <code>#SBATCH --mail-user=you@snu.edu.in</code> || Email address for notifications
|}

== Job Types ==

=== 1. Serial Job (single core) ===

For programs that use only one CPU core. Use the <code>small</code> partition.

<pre>
#!/bin/bash
#SBATCH --job-name=serial_test
#SBATCH --partition=small
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --mem=8G
#SBATCH --time=02:00:00
#SBATCH --output=serial_%j.out
#SBATCH --error=serial_%j.err

module purge
module load intel/oneapi

./my_serial_program input.dat
</pre>

=== 2. Shared Memory Job (OpenMP) ===

For programs parallelized with OpenMP — multiple threads on a single node.

<pre>
#!/bin/bash
#SBATCH --job-name=openmp_job
#SBATCH --partition=medium
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=64
#SBATCH --mem=256G
#SBATCH --time=12:00:00
#SBATCH --output=omp_%j.out
#SBATCH --error=omp_%j.err

module purge
module load intel/oneapi

export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
export OMP_PLACES=cores
export OMP_PROC_BIND=close

./my_openmp_program
</pre>

=== 3. Distributed Memory Job (MPI) ===

For programs parallelized with MPI — multiple processes across one or more nodes.

<pre>
#!/bin/bash
#SBATCH --job-name=mpi_job
#SBATCH --partition=medium
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=64
#SBATCH --mem=256G
#SBATCH --time=7-00:00:00
#SBATCH --output=mpi_%j.out
#SBATCH --error=mpi_%j.err

module purge
module load intel/oneapi
module load mpi/intelmpi

mpirun -n $SLURM_NTASKS ./my_mpi_program input.dat
</pre>

=== 4. Hybrid Job (MPI + OpenMP) ===

For programs that use both MPI (across nodes) and OpenMP (within each node). Requires careful configuration of both thread counts and MPI binding.

<pre>
#!/bin/bash
#SBATCH --job-name=hybrid_job
#SBATCH --partition=large
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=8
#SBATCH --cpus-per-task=16
#SBATCH --mem=256G
#SBATCH --time=14-00:00:00
#SBATCH --output=hybrid_%j.out
#SBATCH --error=hybrid_%j.err

module purge
module load intel/oneapi
module load mpi/intelmpi

export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK

mpirun -n $SLURM_NTASKS ./my_hybrid_program
</pre>

=== 5. GPU Job ===

For CUDA or GPU-accelerated programs. Always use the <code>GPU</code> partition.

<pre>
#!/bin/bash
#SBATCH --job-name=gpu_job
#SBATCH --partition=GPU
#SBATCH --nodes=1
#SBATCH --ntasks=16
#SBATCH --gres=gpu:4
#SBATCH --mem=256G
#SBATCH --time=7-00:00:00
#SBATCH --output=gpu_%j.out
#SBATCH --error=gpu_%j.err

module purge
module load cuda
module load intel/oneapi

./my_cuda_program
</pre>

=== 6. Array Job ===

Run the same program multiple times with different inputs or parameters, all submitted as one job.

<pre>
#!/bin/bash
#SBATCH --job-name=array_job
#SBATCH --partition=small
#SBATCH --nodes=1
#SBATCH --ntasks=8
#SBATCH --mem=32G
#SBATCH --time=04:00:00
#SBATCH --array=1-10
#SBATCH --output=array_%A_%a.out
#SBATCH --error=array_%A_%a.err

# $SLURM_ARRAY_TASK_ID is set to the array index (1..10)
module purge
module load intel/oneapi

./my_program input_${SLURM_ARRAY_TASK_ID}.dat
</pre>

<div style="background:#0d1a2a;border-left:4px solid #00c9ff;border-radius:6px;padding:14px 18px;margin:16px 0;font-size:14px;color:#94a8be;">
💡 <strong>Array job tip:</strong> Use array jobs instead of submitting hundreds of individual <code>sbatch</code> calls. They are tracked as a single entity and are more scheduler-friendly. Each task sees a unique <code>$SLURM_ARRAY_TASK_ID</code> so you can differentiate inputs.
</div>

=== 7. Interactive Session ===

For testing and debugging interactively on a compute node (not the login node):

<pre>
$ srun --pty --partition=small --nodes=1 --ntasks=8 --mem=32G --time=01:00:00 /bin/bash
</pre>

This gives you a shell on a compute node. Exit with <code>exit</code> when done.

== Monitoring Jobs ==

<pre>
# Check the status of your jobs:
$ squeue -u $USER

# Output columns explained:
# JOBID PARTITION NAME USER ST TIME NODES NODELIST
# 12345 medium myjob deepak R 0:23 2 node[01-02]
# Status: R=Running, PD=Pending, CG=Completing, F=Failed

# Get detailed info on a specific job:
$ scontrol show job 12345

# Check completed job stats (CPU time, memory used):
$ sacct -j 12345 --format=JobID,Elapsed,MaxRSS,State
</pre>

== References ==

* [[Magus02/Scheduling_Basics|Scheduling Basics — Theory]]
* [[Magus02/FAQ_Batch_Jobs|FAQ: Batch Jobs — Troubleshooting]]
* [[Magus02/MPI_Usage|MPI Usage]]
* [[Magus02/OpenMP_Usage|OpenMP Usage]]

----
<div style="font-family:monospace;font-size:11px;color:#484f58;text-align:right;">
[[Magus02/Knowledge_Base|↑ Back to Knowledge Base]] · Maintained by HPC Administration · wiki.snu.edu.in
</div>

Magus02/Batch Scheduler - Revision history

Deepak: new