Why MIG is Essential for Magus
Our core HPC environment, Magus02, houses a total of 8 A100 GPU cards. Because our physical GPU footprint is highly concentrated, treating each A100 as an "all-or-nothing" resource creates severe scheduling bottlenecks and immense resource wastage.
Prior to this upgrade, a user running a small script that only required 5GB of memory would lock an entire 80GB card, effectively wasting 75GB of enterprise VRAM and blocking other researchers from working. With only 8 cards total on Magus02, this under-utilization is no longer sustainable. MIG directly solves this problem.
Why Use MIG? (And How It Benefits You)
MIG allows a single A100 physical GPU to be securely partitioned into 7 separate, isolated instances. This directly addresses the cluster bottlenecks discussed in our policy meeting by delivering:
- Drastically Reduced Resource Wastage: Instead of only 8 users locking down the entire cluster, Magus02 can now dynamically host up to 56 individual GPU jobs simultaneously, ensuring no VRAM sits idle.
- Faster Queue Times: You can now run small-to-medium jobs immediately without waiting days in the queue for a full 80GB GPU to become free.
- Hardware-Level Isolation: Each MIG slice has its own dedicated memory, cache, and compute cores. Your job's performance will remain entirely unaffected by other users sharing the same physical card.
- Support for Multi-Process Frameworks: Because each MIG instance operates as a fully distinct GPU device, your multi-process frameworks can seamlessly bind to individual slices. This allows for clean, parallel data-processing tasks without complex internal scheduling.
Updated GPU & CPU Configurations
To align with this shift toward efficiency, we have also lowered the minimum CPU requirement for GPU jobs from 16 cores to 4 cores, allowing you to run much more compact and efficient workloads without wasting CPU cycles.
| Resource Type | VRAM | Compute (SM) | Slurm Directive |
| Full GPU | 80GB | 98 SMs | #SBATCH --gres=gpu:nvidia_a100_80gb_
|
| MIG Slice (Single) | 10GB | 14 SMs | #SBATCH --gres=gpu:nvidia_a100_80gb_
|
Combining & Using Multiple Slices in One Job
If your workload requires more than 10GB of VRAM or needs to scale across multiple independent GPU targets within a single job script, you can request multiple slices simultaneously.
1. Requesting the Slices
To request multiple slices (for example, 2 slices to get 20GB of total VRAM), adjust your Slurm directive count at the end of the string:
#SBATCH --gres=gpu:nvidia_a100_80gb_
2. Utilizing the Slices in Your Job
When you request multiple slices, Slurm will automatically provision them and expose them to your job environment.
- For Multi-Process/Multi-GPU Frameworks: Frameworks like PyTorch (DDP), TensorFlow, or custom multi-process Python scripts will automatically detect these as two distinct GPU devices (Device 0 and Device 1).
- For Parallel Task Execution: You can target specific slices within the same job submission using standard environment variables or Slurm arrays.
Troubleshooting & Device Visibility
Slurm automatically manages and sets up the environment for you. To verify your allocation or troubleshoot device visibility within your job script, you can print the assigned environment variables:
nvidia-smi
Important Tip:' If your code or framework is not detecting the assigned MIG slices, double-check your script to ensure you are not accidentally overwriting or resetting the
$CUDA_VISIBLE_DEVICESvariable during your environment or module setup.
