Bhuvnesh at 05:39, 12 October 2026

2026-10-12T05:39:05Z

Bhuvnesh: Created page with " === Why MIG is Essential for Magus === Our core HPC environment, '''Magus02''', houses a total of '''8 A100 GPU cards'''. Because our physical GPU footprint is highly concent..."

2026-10-12T05:33:38Z

Created page with " === Why MIG is Essential for Magus === Our core HPC environment, '''Magus02''', houses a total of '''8 A100 GPU cards'''. Because our physical GPU footprint is highly concent..."

New page

=== Why MIG is Essential for Magus ===
Our core HPC environment, '''Magus02''', houses a total of '''8 A100 GPU cards'''. Because our physical GPU footprint is highly concentrated, treating each A100 as an "all-or-nothing" resource creates severe scheduling bottlenecks and immense resource wastage.

Prior to this upgrade, a user running a small script that only required 5GB of memory would lock an entire 80GB card, effectively wasting 75GB of enterprise VRAM and blocking other researchers from working. With only 8 cards total on Magus02, this under-utilization is no longer sustainable. MIG directly solves this problem.

=== Why Use MIG? (And How It Benefits You) ===
MIG allows a single A100 physical GPU to be securely partitioned into 7 separate, isolated instances. This directly addresses the cluster bottlenecks discussed in our policy meeting by delivering:

* '''Drastically Reduced Resource Wastage:''' Instead of only 8 users locking down the entire cluster, Magus02 can now dynamically host up to 56 individual GPU jobs simultaneously, ensuring no VRAM sits idle.
* '''Faster Queue Times:''' You can now run small-to-medium jobs immediately without waiting days in the queue for a full 80GB GPU to become free.
* '''Hardware-Level Isolation:''' Each MIG slice has its own dedicated memory, cache, and compute cores. Your job's performance will remain entirely unaffected by other users sharing the same physical card.
* '''Support for Multi-Process Frameworks:''' Because each MIG instance operates as a fully distinct GPU device, your multi-process frameworks can seamlessly bind to individual slices. This allows for clean, parallel data-processing tasks without complex internal scheduling.

=== Updated GPU & CPU Configurations ===
To align with this shift toward efficiency, we have also '''lowered the minimum CPU requirement for GPU jobs from 16 cores to 4 cores''', allowing you to run much more compact and efficient workloads without wasting CPU cycles.
{| class="wikitable" style="color: rgb(34, 34, 34); font-size: small; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; margin-bottom: 32px; font-family: "Google Sans Text", sans-serif; line-height: 1.15; margin-top: 0px;"
| style="margin:0px;border:1px solid;line-height:1.15;" |'''Resource Type'''
| style="margin:0px;border:1px solid;line-height:1.15;" |'''VRAM'''
| style="margin:0px;border:1px solid;line-height:1.15;" |'''Compute (SM)'''
| style="margin:0px;border:1px solid;line-height:1.15;" |'''Slurm Directive'''
|- style="line-height: 1.15; margin-top: 0px;"
| style="margin:0px;border:1px solid;line-height:1.15;" |<span style="line-height: 1.15; margin-top: 0px;">'''Full GPU'''</span>
| style="margin:0px;border:1px solid;line-height:1.15;" |<span style="line-height: 1.15; margin-top: 0px;">80GB</span>
| style="margin:0px;border:1px solid;line-height:1.15;" |<span style="line-height: 1.15; margin-top: 0px;">98 SMs</span>
| style="margin:0px;border:1px solid;line-height:1.15;" |<span style="line-height: 1.15; margin-top: 0px;"><code style="font-family: "Google Sans Text", sans-serif; line-height: 1.15; margin-top: 0px;">#SBATCH --gres=gpu:nvidia_a100_80gb_<wbr />pcie_7g.80gb:1</code></span>
|- style="line-height: 1.15; margin-top: 0px;"
| style="margin:0px;border:1px solid;line-height:1.15;" |<span style="line-height: 1.15; margin-top: 0px;">'''MIG Slice (Single)'''</span>
| style="margin:0px;border:1px solid;line-height:1.15;" |<span style="line-height: 1.15; margin-top: 0px;">10GB</span>
| style="margin:0px;border:1px solid;line-height:1.15;" |<span style="line-height: 1.15; margin-top: 0px;">14 SMs</span>
| style="margin:0px;border:1px solid;line-height:1.15;" |<span style="line-height: 1.15; margin-top: 0px;"><code style="font-family: "Google Sans Text", sans-serif; line-height: 1.15; margin-top: 0px;">#SBATCH --gres=gpu:nvidia_a100_80gb_<wbr />pcie_1g.10gb:1</code></span>
|}

=== Combining & Using Multiple Slices in One Job ===
If your workload requires more than 10GB of VRAM or needs to scale across multiple independent GPU targets within a single job script, you can request multiple slices simultaneously.

==== 1. Requesting the Slices ====
To request multiple slices (for example, 2 slices to get 20GB of total VRAM), adjust your Slurm directive count at the end of the string:

<span style="text-align: center; display: block;">'''''#SBATCH --gres=gpu:nvidia_a100_80gb_<wbr />pcie_1g.10gb:2'''''</span>

==== 2. Utilizing the Slices in Your Job ====
When you request multiple slices, Slurm will automatically provision them and expose them to your job environment.

* '''For Multi-Process/Multi-GPU Frameworks:''' Frameworks like PyTorch (DDP), TensorFlow, or custom multi-process Python scripts will automatically detect these as '''two distinct GPU devices''' (Device 0 and Device 1).
* '''For Parallel Task Execution:''' You can target specific slices within the same job submission using standard environment variables or Slurm arrays.

=== Troubleshooting & Device Visibility ===
Slurm automatically manages and sets up the environment for you. To verify your allocation or troubleshoot device visibility within your job script, you can print the assigned environment variables:
<span style="color: rgb(34, 34, 34); font-size: small; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; font-family: "Google Sans Text", sans-serif; line-height: 1.15; margin-top: 0px;"><span style="line-height: 1.15; margin-top: 0px;"><div style="line-height: 1.15; margin-top: 0px;"><div style="line-height: 1.15; margin-top: 0px;"><div style="line-height: 1.15; margin-top: 0px;">'''''<div style="line-height: 1.15; margin-top: 0px;"><span style="line-height: 1.15; margin-top: 0px;">echo</span> <span style="line-height: 1.15; margin-top: 0px;">"Allocated CUDA Devices: <span style="line-height: 1.15; margin-top: 0px;">$CUDA_VISIBLE_DEVICES</span>"</span></div>
<code role="text" style="font-family: "Google Sans Text", sans-serif; line-height: 1.15; margin-top: 0px;">nvidia-smi
</code>
'''''</div></div></div></span></span><blockquote style="color: rgb(34, 34, 34); font-size: small; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; font-family: "Google Sans Text", sans-serif; line-height: 1.15; margin-top: 0px;">https://fonts.gstatic.com/s/e/notoemoji/17.0/1f4a1/32.png '''Important Tip:''' If your code or framework is not detecting the assigned MIG slices, double-check your script to ensure you are not accidentally overwriting or resetting the <code style="font-family: "Google Sans Text", sans-serif; line-height: 1.15; margin-top: 0px;">$CUDA_VISIBLE_DEVICES</code> variable during your environment or module setup.</blockquote>

@@ Line 51: / Line 51: @@
   <code role="text" style="font-family: &quot;Google Sans Text&quot;, sans-serif; line-height: 1.15; margin-top: 0px;">nvidia-smi
   </code>
-'''''</div></div></div></span></span><blockquote style="color: rgb(34, 34, 34); font-size: small; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; font-family: &quot;Google Sans Text&quot;, sans-serif; line-height: 1.15; margin-top: 0px;">https://fonts.gstatic.com/s/e/notoemoji/17.0/1f4a1/32.png '''Important Tip:''' If your code or framework is not detecting the assigned MIG slices, double-check your script to ensure you are not accidentally overwriting or resetting the <code style="font-family: &quot;Google Sans Text&quot;, sans-serif; line-height: 1.15; margin-top: 0px;">$CUDA_VISIBLE_DEVICES</code> variable during your environment or module setup.</blockquote>
+'''''</div></div></div></span></span><blockquote style="color: rgb(34, 34, 34); font-size: small; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; white-space: normal; background-color: rgb(255, 255, 255); text-decoration-thickness: initial; text-decoration-style: initial; text-decoration-color: initial; font-family: &quot;Google Sans Text&quot;, sans-serif; line-height: 1.15; margin-top: 0px;">''Important Tip:'<nowiki/>'' If your code or framework is not detecting the assigned MIG slices, double-check your script to ensure you are not accidentally overwriting or resetting the <code style="font-family: &quot;Google Sans Text&quot;, sans-serif; line-height: 1.15; margin-top: 0px;">$CUDA_VISIBLE_DEVICES</code> variable during your environment or module setup.</blockquote>

GPU MIG SLICE - Revision history

Bhuvnesh at 05:39, 12 October 2026

Bhuvnesh: Created page with " === Why MIG is Essential for Magus === Our core HPC environment, '''Magus02''', houses a total of '''8 A100 GPU cards'''. Because our physical GPU footprint is highly concent..."