Deepak: new

2026-04-20T06:39:08Z

new

New page

{{DISPLAYTITLE:Scheduling Basics — Magus02 HPC}}
<div style="background:#0d1b2a;border:1px solid #1e3a5a;border-radius:8px;padding:12px 18px;margin-bottom:20px;font-size:13px;font-family:monospace;color:#5ba3d9;">
📍 [[Magus02/Knowledge_Base|Magus02 Knowledge Base]] → Basics → Scheduling Basics
</div>

This page explains the fundamental concepts behind batch scheduling on HPC clusters. For specific Slurm commands and job types, see [[Magus02/Batch_Scheduler|Batch Scheduler]].

__TOC__

== What is a Batch Scheduler? ==

On Magus02, users do not run calculations directly and interactively on the compute nodes (as they would on a laptop). Instead, they submit '''batch jobs''' to the '''Slurm scheduler'''.

The scheduler:
* Stores submitted jobs in a queue
* Evaluates each job's resource requirements and priority
* Distributes jobs to suitable compute nodes when resources are available
* Manages all 57+ compute nodes making up 98%+ of the cluster

The '''login nodes''' are for compiling code, editing files, and submitting jobs. The '''compute nodes''' (accessed only through the scheduler) are where actual computation happens.

<div style="background:#2a1a00;border-left:4px solid #f0c040;border-radius:6px;padding:14px 18px;margin:16px 0;font-size:14px;color:#d4b870;">
⚠️ <strong>Do not run compute jobs on login nodes.</strong> Heavy workloads on login nodes slow the system for all users and may result in your processes being killed. Always use Slurm for anything that takes more than a few seconds.
</div>

== How a Batch Job Works ==

# You write a '''jobscript''' — a shell script containing your commands and resource requests
# You submit it with <code>sbatch jobscript.sh</code>
# Slurm adds it to a queue and waits for enough free resources
# When resources become available, Slurm runs your job on compute nodes
# Output is written to files you specified in the script

You specify three key resources:

{| class="wikitable" style="width:100%;"
|-
! Resource !! Slurm Flag !! Description
|-
| Time || <code>--time=HH:MM:SS</code> || Maximum runtime. Job is killed if it exceeds this.
|-
| Cores || <code>--ntasks</code> or <code>--cpus-per-task</code> || Number of CPU cores needed.
|-
| Memory || <code>--mem=XG</code> or <code>--mem-per-cpu=XG</code> || RAM needed. Job may crash or not start if underspecified.
|}

== Scheduling Algorithms ==

Modern schedulers like Slurm use a combination of strategies:

=== First Come, First Served (FCFS) ===

Jobs run in the order they were submitted. Simple and fair, but small jobs can wait behind large ones unnecessarily.

=== Shortest Job First ===

Jobs with shorter declared runtimes get priority. Very long-running jobs may be delayed significantly.

=== Backfilling ===

Slurm's default strategy. It maintains FCFS priority but allows shorter jobs to ''fill gaps'' in the schedule without delaying higher-priority jobs. This means short jobs typically see short wait times even when the cluster is busy.

To benefit from backfilling, '''request an accurate runtime''' — not too long (wastes priority) and not too short (job gets killed before finishing).

== Magus02 Queue Structure ==

{| class="wikitable" style="width:100%;font-size:13px;"
|-
! style="background:#0d1b2a;color:#94a8be;" | Queue
! style="background:#0d1b2a;color:#94a8be;" | Processor
! style="background:#0d1b2a;color:#94a8be;" | Min Cores
! style="background:#0d1b2a;color:#94a8be;" | Max Cores
! style="background:#0d1b2a;color:#94a8be;" | Max Nodes
! style="background:#0d1b2a;color:#94a8be;" | Wall Time
! style="background:#0d1b2a;color:#94a8be;" | Max Cores/User
! style="background:#0d1b2a;color:#94a8be;" | Max Pending
! style="background:#0d1b2a;color:#94a8be;" | Max Running
|-
| <code>medium</code> || AMD EPYC 7763 || 32 || 128 || 77 || 1 week || 512 || 2 || 2
|-
| <code>small</code> || AMD EPYC 7043 || 8 || 32 || 16 || 1 month || 128 || 2 || 4
|-
| <code>high_mem</code> || AMD EPYC 7543 || 32 || 256 || 12 || 2 weeks || 1024 || 2 || 2
|-
| <code>large</code> || AMD EPYC 7763 || 64 || 256 || 24 || 2 weeks || 1024 || 2 || 2
|-
| <code>GPU</code> || Intel Xeon Platinum 8368 || 16 || 16 || 4 || 1 week || 32 || 2 || 2
|-
| <code>low/long</code> || AMD EPYC 7763 || 32 || 1024 || All || 1 month || 1024 || 2 || 8
|}

<div style="background:#0d1a0d;border-left:4px solid #39d353;border-radius:6px;padding:14px 18px;margin:16px 0;font-size:14px;color:#80b880;">
✅ <strong>Queue selection guide:</strong>
* Testing / debugging → <strong>small</strong>
* Standard multi-node runs → <strong>medium</strong>
* Large production jobs → <strong>large</strong>
* Gaussian / large in-memory apps → <strong>high_mem</strong>
* GPU workloads (CUDA) → <strong>GPU</strong>
* Long, low-priority jobs → <strong>low/long</strong>
</div>

== References ==

* [[Magus02/Batch_Scheduler|Batch Scheduler — Job Submission Guide]]
* [[Magus02/FAQ_Batch_Jobs|FAQ: Batch Jobs]]
* [[Magus02|Magus02 Main Page — Queue Details]]

----
<div style="font-family:monospace;font-size:11px;color:#484f58;text-align:right;">
[[Magus02/Knowledge_Base|↑ Back to Knowledge Base]] · Maintained by HPC Administration · wiki.snu.edu.in
</div>

Magus02/Scheduling Basics - Revision history

Deepak: new