Magus02 — HPC Knowledge Base

(New Page)
Tag: 2017 source edit
 
(redesign)
Tag: 2017 source edit
 
Line 1: Line 1:
−
{{DISPLAYTITLE:Magus02 — HPC Cluster}}
+
{{DISPLAYTITLE:Magus02 — HPC Knowledge Base}}
−
__TOC__
+
__NOTOC__
  
−
<div style="background:linear-gradient(135deg,#0d1b2a 0%,#1a2a3f 100%);border:1px solid #2a4a6b;border-radius:10px;padding:28px 32px;margin-bottom:28px;">
+
<div style="background:linear-gradient(135deg,#0d1b2a 0%,#1a2a3f 100%);border:1px solid #2a4a6b;border-radius:10px;padding:30px 36px;margin-bottom:28px;">
 
<div style="font-family:monospace;font-size:11px;color:#5ba3d9;letter-spacing:2px;text-transform:uppercase;margin-bottom:10px;">Shiv Nadar Institution of Eminence — HPC Infrastructure</div>
 
<div style="font-family:monospace;font-size:11px;color:#5ba3d9;letter-spacing:2px;text-transform:uppercase;margin-bottom:10px;">Shiv Nadar Institution of Eminence — HPC Infrastructure</div>
−
<div style="font-size:36px;font-weight:900;color:#ffffff;letter-spacing:-1px;margin-bottom:10px;">Magus02</div>
+
<div style="font-size:40px;font-weight:900;color:#ffffff;letter-spacing:-1px;line-height:1;margin-bottom:12px;">Magus02 HPC<br>Knowledge Base</div>
−
<div style="font-size:15px;color:#94a8be;max-width:620px;line-height:1.7;margin-bottom:20px;">The primary High Performance Computing cluster at SNU, providing researchers and students with parallel compute resources for scientific simulation, deep learning, computational chemistry, and data-intensive workflows.</div>
+
<div style="font-size:15px;color:#94a8be;max-width:680px;line-height:1.7;margin-bottom:24px;">Your complete reference for accessing, using, and getting the most out of the Magus02 High Performance Computing cluster at SNU. Whether you are a first-time user or an experienced HPC programmer, this wiki has you covered.</div>
 +
<div style="display:flex;gap:12px;flex-wrap:wrap;">
 +
<span style="background:#00c9ff;color:#060e18;font-family:monospace;font-size:12px;font-weight:700;padding:6px 14px;border-radius:5px;">HOST: magus02.snu.edu.in</span>
 +
<span style="background:rgba(57,211,83,0.15);color:#39d353;font-family:monospace;font-size:12px;font-weight:700;padding:6px 14px;border-radius:5px;border:1px solid rgba(57,211,83,0.3);">● SYSTEM OPERATIONAL</span>
 +
<span style="background:rgba(0,201,255,0.1);color:#00c9ff;font-family:monospace;font-size:12px;padding:6px 14px;border-radius:5px;border:1px solid rgba(0,201,255,0.2);">SCHEDULER: Slurm</span>
 +
</div>
 +
</div>
  
−
{| style="border-collapse:separate;border-spacing:16px 0;"
+
== 🚀 Quick Navigation ==
 +
 
 +
{| style="width:100%;border-collapse:separate;border-spacing:14px;"
 
|-
 
|-
−
| style="background:#162131;border:1px solid #2a4060;border-radius:8px;padding:14px 20px;text-align:center;" |
+
| colspan="2" style="background:#0a1520;border:2px solid #00c9ff;border-radius:10px;padding:20px 24px;" |
−
<div style="font-size:26px;font-weight:900;color:#00c9ff;">9064</div>
+
<div style="font-size:13px;font-family:monospace;color:#5ba3d9;letter-spacing:1px;text-transform:uppercase;margin-bottom:8px;">⚡ New to Magus02? Start here</div>
−
<div style="font-size:10px;font-family:monospace;color:#5a7a9a;text-transform:uppercase;letter-spacing:1px;">Total Cores</div>
+
<div style="font-size:16px;font-weight:700;color:#e6edf3;margin-bottom:6px;">[[Magus02/Getting_Started|→ Getting Started Guide]]</div>
−
| style="background:#162131;border:1px solid #2a4060;border-radius:8px;padding:14px 20px;text-align:center;" |
+
<div style="font-size:14px;color:#7a94aa;">Everything from account access and SSH login to running your first job — in one guide.</div>
−
<div style="font-size:26px;font-weight:900;color:#00c9ff;">238 TFlops</div>
+
|-
−
<div style="font-size:10px;font-family:monospace;color:#5a7a9a;text-transform:uppercase;letter-spacing:1px;">Peak Performance</div>
+
| style="width:50%;background:#0d1b2a;border:1px solid #1e3a5a;border-radius:10px;padding:20px 22px;vertical-align:top;" |
−
| style="background:#162131;border:1px solid #2a4060;border-radius:8px;padding:14px 20px;text-align:center;" |
+
<div style="font-size:12px;font-family:monospace;color:#5ba3d9;letter-spacing:1px;text-transform:uppercase;margin-bottom:12px;">📘 Basics</div>
−
<div style="font-size:26px;font-weight:900;color:#00c9ff;">57</div>
+
<div style="font-size:13px;line-height:2.2;color:#94a8be;">
−
<div style="font-size:10px;font-family:monospace;color:#5a7a9a;text-transform:uppercase;letter-spacing:1px;">Compute Nodes</div>
+
[[Magus02/Getting_Started|Getting Started]] · First steps on Magus02<br>
−
| style="background:#162131;border:1px solid #2a4060;border-radius:8px;padding:14px 20px;text-align:center;" |
+
[[Magus02/Shell|Linux Shell]] · Commands and filesystem navigation<br>
−
<div style="font-size:26px;font-weight:900;color:#00c9ff;">750 TB</div>
+
[[Magus02/SSH|SSH Access]] · Connecting securely to the cluster<br>
−
<div style="font-size:10px;font-family:monospace;color:#5a7a9a;text-transform:uppercase;letter-spacing:1px;">Storage</div>
+
[[Magus02/File_Transfer|File Transfer]] · scp, rsync, WinSCP, FileZilla<br>
 +
[[Magus02/Modules|Environment Modules]] · Loading software on HPC<br>
 +
[[Magus02/Scheduling_Basics|Scheduling Basics]] · How the batch system works
 +
</div>
 +
| style="width:50%;background:#0d1b2a;border:1px solid #1e3a5a;border-radius:10px;padding:20px 22px;vertical-align:top;" |
 +
<div style="font-size:12px;font-family:monospace;color:#5ba3d9;letter-spacing:1px;text-transform:uppercase;margin-bottom:12px;">⚙️ HPC User Guide</div>
 +
<div style="font-size:13px;line-height:2.2;color:#94a8be;">
 +
[[Magus02/Compiler|Compilers]] · GCC, Intel, LLVM, PGI<br>
 +
[[Magus02/Batch_Scheduler|Batch Scheduler]] · Submitting and managing jobs<br>
 +
[[Magus02/OpenMP_Usage|OpenMP Usage]] · Shared-memory parallelism<br>
 +
[[Magus02/MPI_Usage|MPI Usage]] · Distributed-memory parallelism<br>
 +
[[Magus02/NUMA|NUMA Architecture]] · Memory locality and pinning<br>
 +
[[Magus02/FAQ_Batch_Jobs|FAQ: Batch Jobs]] · Common problems and solutions
 +
</div>
 
|}
 
|}
−
</div>
 
  
−
== What is HPC? ==
+
== 🖥️ About Magus02 ==
  
−
High Performance Computing (HPC) is the ability to process data and perform complex calculations at high speeds. To put it in perspective: a laptop or desktop with a 3 GHz processor can perform around 3 billion calculations per second. HPC solutions can perform '''quadrillions of calculations per second''' — far exceeding anything achievable on a standard workstation.
+
Magus02 is the primary HPC cluster at Shiv Nadar Institution of Eminence, Delhi NCR, hosted within the Department of Chemical Engineering. It provides researchers, faculty, and students with large-scale parallel computing resources for a wide range of workloads including computational chemistry, fluid dynamics, machine learning, and data analysis.
  
−
HPC is specifically needed for the following reasons:
+
=== Cluster at a Glance ===
−
 
 
−
* It paves the way for '''new innovations''' in science, technology, business, and academia.
 
−
* It '''improves processing speeds''', which can be critical for computing operations, applications, and workloads.
 
−
* It helps lay the foundation for a '''reliable, fast IT infrastructure''' that can store, process, and analyse massive amounts of data for various applications.
 
−
 
 
−
=== Quick Specifications ===
 
  
 
{| class="wikitable" style="width:100%;text-align:center;"
 
{| class="wikitable" style="width:100%;text-align:center;"
 
|-
 
|-
−
! style="background:#0d1b2a;color:#00c9ff;font-family:monospace;font-size:11px;letter-spacing:1px;" | TOTAL CORES (AMD)
+
! style="background:#0d1b2a;color:#00c9ff;font-family:monospace;font-size:11px;letter-spacing:1px;" | TOTAL CORES
 
! style="background:#0d1b2a;color:#00c9ff;font-family:monospace;font-size:11px;letter-spacing:1px;" | PEAK PERFORMANCE
 
! style="background:#0d1b2a;color:#00c9ff;font-family:monospace;font-size:11px;letter-spacing:1px;" | PEAK PERFORMANCE
 
! style="background:#0d1b2a;color:#00c9ff;font-family:monospace;font-size:11px;letter-spacing:1px;" | COMPUTE NODES
 
! style="background:#0d1b2a;color:#00c9ff;font-family:monospace;font-size:11px;letter-spacing:1px;" | COMPUTE NODES
−
! style="background:#0d1b2a;color:#00c9ff;font-family:monospace;font-size:11px;letter-spacing:1px;" | LOGIN NODES
+
! style="background:#0d1b2a;color:#00c9ff;font-family:monospace;font-size:11px;letter-spacing:1px;" | GPU NODES
−
! style="background:#0d1b2a;color:#00c9ff;font-family:monospace;font-size:11px;letter-spacing:1px;" | MASTER NODES
+
! style="background:#0d1b2a;color:#00c9ff;font-family:monospace;font-size:11px;letter-spacing:1px;" | TOTAL RAM
−
! style="background:#0d1b2a;color:#00c9ff;font-family:monospace;font-size:11px;letter-spacing:1px;" | HIGH MEM NODES (SSD)
+
! style="background:#0d1b2a;color:#00c9ff;font-family:monospace;font-size:11px;letter-spacing:1px;" | STORAGE
 
|-
 
|-
−
| '''9064''' || '''238 TFlops''' || '''57''' || '''2''' || '''2''' || '''12'''
+
| '''9,064''' || '''238 TFlops''' || '''57''' || '''2 (Tesla A100×4)''' || '''~42 TB''' || '''750 TB'''
 
|}
 
|}
−
 
−
<div style="background:#0d2a1a;border-left:4px solid #39d353;border-radius:6px;padding:14px 18px;margin:16px 0;font-size:14px;color:#c0d8c8;">
 
−
💡 <strong>Memory per Core:</strong> Compute Nodes → <strong>4 GB/core</strong> (total ~29,184 GB) &nbsp;|&nbsp; High Memory Nodes → <strong>16 GB/core</strong> (total ~12,288 GB). Use the <code>high_mem</code> queue for memory-bound applications such as Gaussian.
 
−
</div>
 
−
 
−
== Compute Specification ==
 
  
 
{| class="wikitable sortable" style="width:100%;"
 
{| class="wikitable sortable" style="width:100%;"
Line 58: Line 66:
 
! style="background:#0d1b2a;color:#94a8be;" | Node Type
 
! style="background:#0d1b2a;color:#94a8be;" | Node Type
 
! style="background:#0d1b2a;color:#94a8be;" | Processor
 
! style="background:#0d1b2a;color:#94a8be;" | Processor
−
! style="background:#0d1b2a;color:#94a8be;" | RAM
+
! style="background:#0d1b2a;color:#94a8be;" | RAM / Node
−
! style="background:#0d1b2a;color:#94a8be;" | Cores
+
! style="background:#0d1b2a;color:#94a8be;" | Cores / Node
 
! style="background:#0d1b2a;color:#94a8be;" | GPU
 
! style="background:#0d1b2a;color:#94a8be;" | GPU
−
! style="background:#0d1b2a;color:#94a8be;" | No. of Nodes
+
! style="background:#0d1b2a;color:#94a8be;" | Count
 
! style="background:#0d1b2a;color:#94a8be;" | Total RAM
 
! style="background:#0d1b2a;color:#94a8be;" | Total RAM
 
! style="background:#0d1b2a;color:#94a8be;" | Total Cores
 
! style="background:#0d1b2a;color:#94a8be;" | Total Cores
 
|-
 
|-
−
| <span style="background:#0d2535;color:#00c9ff;font-family:monospace;font-size:12px;padding:3px 9px;border-radius:4px;border:1px solid #1a4060;">Compute Node</span>
+
| <span style="background:#0d2535;color:#00c9ff;font-family:monospace;font-size:12px;padding:2px 8px;border-radius:4px;">Compute Node</span>
−
| <code>AMD EPYC 7763</code> || 512 GB || 128 || — || '''57''' || 29,184 GB || 7,296
+
| <code>AMD EPYC 7763</code> || 512 GB || 128 || — || 57 || 29,184 GB || 7,296
 
|-
 
|-
−
| <span style="background:#2a2000;color:#f0c040;font-family:monospace;font-size:12px;padding:3px 9px;border-radius:4px;border:1px solid #504000;">High Memory Node</span>
+
| <span style="background:#2a2000;color:#f0c040;font-family:monospace;font-size:12px;padding:2px 8px;border-radius:4px;">High Memory Node</span>
−
| <code>AMD EPYC 7543</code> || 1024 GB || 64 || — || '''12''' || 12,288 GB || 768
+
| <code>AMD EPYC 7543</code> || 1,024 GB || 64 || — || 12 || 12,288 GB || 768
 
|-
 
|-
−
| <span style="background:#0d2a10;color:#39d353;font-family:monospace;font-size:12px;padding:3px 9px;border-radius:4px;border:1px solid #1a5020;">GPU Node</span>
+
| <span style="background:#0d2a10;color:#39d353;font-family:monospace;font-size:12px;padding:2px 8px;border-radius:4px;">GPU Node</span>
−
| <code>Intel Xeon Platinum 8368</code> || 512 GB || 64 || Tesla A100 ×4 || '''2''' || 1,024 GB || 128
+
| <code>Intel Xeon Platinum 8368</code> || 512 GB || 64 || Tesla A100 ×4 || 2 || 1,024 GB || 128
 
|-
 
|-
−
| style="background:#1a1a1a;font-family:monospace;font-size:12px;color:#666;" | '''Totals'''
+
| <span style="background:#2a1010;color:#f85149;font-family:monospace;font-size:12px;padding:2px 8px;border-radius:4px;">Storage</span>
−
| || || || || style="color:#00c9ff;font-weight:bold;" | 71 nodes || style="color:#00c9ff;font-weight:bold;" | ~42,496 GB || style="color:#00c9ff;font-weight:bold;" | 8,192
+
| colspan="7" | <code>750 TB</code> Parallel Filesystem
−
|-
 
−
| <span style="background:#2a1010;color:#f85149;font-family:monospace;font-size:12px;padding:3px 9px;border-radius:4px;border:1px solid #502020;">Storage</span>
 
−
| colspan="7" | <code>750 TB</code> — Parallel Filesystem
 
 
|}
 
|}
  
−
<div style="background:#2a1a00;border-left:4px solid #f0c040;border-radius:6px;padding:14px 18px;margin:16px 0;font-size:14px;color:#d4b870;">
+
=== Queue Summary ===
−
⚠️ <strong>High Memory Nodes</strong> are intended for applications that require large in-memory datasets but do not scale well across nodes (e.g., <em>Gaussian</em>). Submit to the <code>high_mem</code> queue and use <code>#SBATCH --mem</code> to request appropriate memory.
 
−
</div>
 
  
−
== How to Connect to Magus02 ==
+
{| class="wikitable" style="width:100%;font-size:13px;"
−
 
+
|-
−
Magus02 supports SSH-based access. Three connection methods are available:
+
! style="background:#0d1b2a;color:#94a8be;" | Queue
−
 
+
! style="background:#0d1b2a;color:#94a8be;" | Node Type
−
{| style="width:100%;border-collapse:separate;border-spacing:12px 0;"
+
! style="background:#0d1b2a;color:#94a8be;" | Max Cores/Job
 +
! style="background:#0d1b2a;color:#94a8be;" | Max Nodes
 +
! style="background:#0d1b2a;color:#94a8be;" | Wall Time
 +
! style="background:#0d1b2a;color:#94a8be;" | Use Case
 +
|-
 +
| <span style="background:#0d2535;color:#00c9ff;font-family:monospace;font-size:11px;padding:2px 8px;border-radius:4px;">medium</span> || Compute || 128 || 77 || 1 week || General multi-node jobs
 +
|-
 +
| <span style="background:#0d2a10;color:#39d353;font-family:monospace;font-size:11px;padding:2px 8px;border-radius:4px;">small</span> || Compute || 32 || 16 || 1 month || Testing and small runs
 +
|-
 +
| <span style="background:#2a2000;color:#f0c040;font-family:monospace;font-size:11px;padding:2px 8px;border-radius:4px;">high_mem</span> || High Memory || 256 || 12 || 2 weeks || Gaussian, large in-memory jobs
 +
|-
 +
| <span style="background:#2a1010;color:#f85149;font-family:monospace;font-size:11px;padding:2px 8px;border-radius:4px;">large</span> || Compute || 256 || 24 || 2 weeks || Large parallel production runs
 +
|-
 +
| <span style="background:#1a0a30;color:#b392f0;font-family:monospace;font-size:11px;padding:2px 8px;border-radius:4px;">GPU</span> || GPU || 16 || 4 || 1 week || CUDA/GPU workloads
 
|-
 
|-
−
| style="width:33%;background:#0d1b2a;border:1px solid #1e3a5a;border-radius:10px;padding:20px;vertical-align:top;" |
+
| <span style="background:#1a1a1a;color:#8b949e;font-family:monospace;font-size:11px;padding:2px 8px;border-radius:4px;">low/long</span> || Compute || 1024 || All || 1 month || Long, low-priority jobs
−
<div style="font-size:22px;margin-bottom:10px;">🌐</div>
 
−
<div style="font-weight:700;font-size:15px;margin-bottom:8px;color:#e6edf3;">Web Browser</div>
 
−
<div style="font-size:13px;color:#7a94aa;margin-bottom:12px;">Visit the login portal directly from any browser on the SNU network.</div>
 
−
<div style="background:#060e18;border:1px solid #1a3050;border-radius:5px;padding:8px 12px;font-family:monospace;font-size:12px;color:#79c0ff;">magus02.snu.edu.in</div>
 
−
| style="width:33%;background:#0d1b2a;border:1px solid #1e3a5a;border-radius:10px;padding:20px;vertical-align:top;" |
 
−
<div style="font-size:22px;margin-bottom:10px;">🐧</div>
 
−
<div style="font-weight:700;font-size:15px;margin-bottom:8px;color:#e6edf3;">Linux / macOS</div>
 
−
<div style="font-size:13px;color:#7a94aa;margin-bottom:12px;">Use the built-in SSH client from your terminal. Default port is 22.</div>
 
−
<div style="background:#060e18;border:1px solid #1a3050;border-radius:5px;padding:8px 12px;font-family:monospace;font-size:12px;color:#79c0ff;">ssh [email protected]</div>
 
−
| style="width:33%;background:#0d1b2a;border:1px solid #1e3a5a;border-radius:10px;padding:20px;vertical-align:top;" |
 
−
<div style="font-size:22px;margin-bottom:10px;">🪟</div>
 
−
<div style="font-weight:700;font-size:15px;margin-bottom:8px;color:#e6edf3;">Windows (PuTTY)</div>
 
−
<div style="font-size:13px;color:#7a94aa;margin-bottom:12px;">Download PuTTY and connect using the hostname and port below.</div>
 
−
<div style="background:#060e18;border:1px solid #1a3050;border-radius:5px;padding:8px 12px;font-family:monospace;font-size:12px;color:#79c0ff;">Host: magus02.snu.edu.in<br>Port: 22</div>
 
 
|}
 
|}
  
−
<pre style="background:#0d1117;border:1px solid #21293a;border-radius:8px;padding:18px;font-family:monospace;font-size:13px;color:#e6edf3;">
+
→ See [[Magus02|Magus02 Cluster Page]] for full queue details, connection instructions, and application list.
−
# Connect from Linux / macOS terminal:
 
−
$ ssh [email protected]          # default port 22
 
−
 
 
−
# Specify port explicitly:
 
−
$ ssh -p 22 [email protected]
 
−
</pre>
 
−
 
 
−
== How to Reset Password on Magus02 ==
 
−
 
 
−
<div style="counter-reset:steps;">
 
−
 
 
−
<div style="display:flex;gap:20px;padding:20px 0;border-bottom:1px solid #1e2d3d;">
 
−
<div style="min-width:40px;height:40px;background:#0d1b2a;border:1px solid #2a4060;border-radius:50%;display:flex;align-items:center;justify-content:center;font-family:monospace;font-weight:700;color:#00c9ff;font-size:13px;flex-shrink:0;text-align:center;line-height:40px;">01</div>
 
−
<div>
 
−
<div style="font-weight:700;font-size:15px;margin-bottom:6px;margin-top:8px;">Log in to your Magus02 account</div>
 
−
<div style="font-size:14px;color:#7a94aa;">Connect via SSH using your existing credentials. If you cannot log in, contact the HPC administrator to reset your account.</div>
 
−
</div>
 
−
</div>
 
−
 
 
−
<div style="display:flex;gap:20px;padding:20px 0;">
 
−
<div style="min-width:40px;height:40px;background:#0d1b2a;border:1px solid #2a4060;border-radius:50%;display:flex;align-items:center;justify-content:center;font-family:monospace;font-weight:700;color:#00c9ff;font-size:13px;flex-shrink:0;text-align:center;line-height:40px;">02</div>
 
−
<div>
 
−
<div style="font-weight:700;font-size:15px;margin-bottom:6px;margin-top:8px;">Run the <code>passwd</code> command</div>
 
−
<div style="font-size:14px;color:#7a94aa;">Once logged in, use the standard passwd utility. You will be prompted for your current password, then asked to enter and confirm the new one.</div>
 
−
<pre style="background:#0d1117;border:1px solid #21293a;border-radius:8px;padding:16px;font-family:monospace;font-size:13px;color:#e6edf3;margin-top:12px;">
 
−
[user@magus02 ~]$ passwd
 
−
Changing password for user user.
 
−
Current password: ••••••••
 
−
New password: ••••••••
 
−
Retype new password: ••••••••
 
−
passwd: all authentication tokens updated successfully.
 
−
</pre>
 
−
</div>
 
−
</div>
 
−
 
 
−
</div>
 
−
 
 
−
<div style="background:#0d1a2a;border-left:4px solid #00c9ff;border-radius:6px;padding:14px 18px;margin:16px 0;font-size:14px;color:#94a8be;">
 
−
🔐 <strong>Password Policy:</strong> Minimum 8 characters including uppercase, lowercase, numbers, and special characters. Shared or weak passwords are strictly prohibited.
 
−
</div>
 
−
 
 
−
== How to Connect to Magus from Outside SNU Network ==
 
−
 
 
−
<div style="background:#2a1e00;border:1px solid #5a4010;border-radius:10px;padding:22px 26px;margin:16px 0;display:flex;gap:18px;align-items:flex-start;">
 
−
<div style="font-size:30px;flex-shrink:0;">🔒</div>
 
−
<div>
 
−
<div style="font-weight:700;font-size:16px;color:#f0c040;margin-bottom:8px;">VPN Required for External Access</div>
 
−
<div style="font-size:14px;color:#c8a850;line-height:1.7;">To gain access to Magus02 from outside the SNU network, a <strong>VPN connection is mandatory</strong>. To acquire VPN access, submit a VPN Request Form or scan the QR barcode provided by the HPC administration team. Once connected, SSH works as normal.</div>
 
−
</div>
 
−
</div>
 
−
 
 
−
<pre style="background:#0d1117;border:1px solid #21293a;border-radius:8px;padding:18px;font-family:monospace;font-size:13px;color:#e6edf3;">
 
−
# Step 1: Connect to SNU VPN using the provided VPN client
 
−
# Step 2: Once VPN is active, SSH as usual:
 
− 
−
</pre>
 
−
 
 
−
== How to Transfer Data using WinSCP (Windows Client) ==
 
−
 
 
−
WinSCP is a free Windows SFTP/SCP client that provides a graphical interface for transferring files between your Windows machine and the cluster.
 
−
 
 
−
; Step 1 — Open WinSCP and fill in connection details, then click Login
 
−
: Launch WinSCP. In the Login dialog enter:
 
−
:* '''File protocol:''' SFTP
 
−
:* '''Host name:''' <code>magus02.snu.edu.in</code>
 
−
:* '''Port number:''' <code>22</code>
 
−
:* '''User name:''' your username
 
−
:* '''Password:''' your password
 
−
: Click the '''Login''' button.
 
−
 
 
−
; Step 2 — Click Yes on the Warning dialog
 
−
: On first connection, WinSCP will display a host key warning. Click '''Yes''' to accept and proceed.
 
−
 
 
−
; Step 3 — Transfer data using the Key Panel
 
−
: Once connected, you will see a dual-panel view: local files on the left, remote cluster files on the right.
 
−
:* '''To upload:''' Right-click a local file → Upload
 
−
:* '''To download:''' Right-click a remote file → Download
 
−
: Drag-and-drop is also supported between panels.
 
−
 
 
−
<div style="background:#0d1a2a;border-left:4px solid #00c9ff;border-radius:6px;padding:14px 18px;margin:16px 0;font-size:14px;color:#94a8be;">
 
−
💡 <strong>Linux / macOS alternative:</strong> Use <code>scp</code> or <code>rsync</code> from the terminal.<br><code>scp myfile.tar.gz [email protected].in:~/</code>
 
−
</div>
 
  
−
== Applications on Magus ==
+
== 📚 Documentation Sections ==
  
−
The following scientific applications are pre-installed on Magus02. Use the module system to load the appropriate environment before submitting jobs.
+
=== Basics (New Users) ===
  
 
{| style="width:100%;border-collapse:separate;border-spacing:10px;"
 
{| style="width:100%;border-collapse:separate;border-spacing:10px;"
 
|-
 
|-
−
| style="background:#0d1b2a;border:1px solid #1e3a5a;border-radius:8px;padding:12px 16px;font-weight:600;" | 🔷 VASP
+
| style="background:#0d1b2a;border:1px solid #1e3a5a;border-radius:8px;padding:16px 18px;vertical-align:top;width:33%;" |
−
| style="background:#0d1b2a;border:1px solid #1e3a5a;border-radius:8px;padding:12px 16px;font-weight:600;" | 🔷 Quantum ESPRESSO
+
<div style="font-size:18px;margin-bottom:8px;">🖥️</div>
−
| style="background:#0d1b2a;border:1px solid #1e3a5a;border-radius:8px;padding:12px 16px;font-weight:600;" | 🔷 Gaussian
+
<div style="font-weight:700;font-size:14px;margin-bottom:6px;color:#e6edf3;">[[Magus02/Getting_Started|Getting Started]]</div>
 +
<div style="font-size:13px;color:#7a94aa;">Access, login, VPN, first steps on the cluster.</div>
 +
| style="background:#0d1b2a;border:1px solid #1e3a5a;border-radius:8px;padding:16px 18px;vertical-align:top;width:33%;" |
 +
<div style="font-size:18px;margin-bottom:8px;">⌨️</div>
 +
<div style="font-weight:700;font-size:14px;margin-bottom:6px;color:#e6edf3;">[[Magus02/Shell|Linux Shell]]</div>
 +
<div style="font-size:13px;color:#7a94aa;">Essential commands for navigation, files, and utilities.</div>
 +
| style="background:#0d1b2a;border:1px solid #1e3a5a;border-radius:8px;padding:16px 18px;vertical-align:top;width:33%;" |
 +
<div style="font-size:18px;margin-bottom:8px;">🔒</div>
 +
<div style="font-weight:700;font-size:14px;margin-bottom:6px;color:#e6edf3;">[[Magus02/SSH|SSH Access]]</div>
 +
<div style="font-size:13px;color:#7a94aa;">Connecting from Linux, macOS, and Windows.</div>
 
|-
 
|-
−
| style="background:#0d1b2a;border:1px solid #1e3a5a;border-radius:8px;padding:12px 16px;font-weight:600;" | 🔷 Siesta
+
| style="background:#0d1b2a;border:1px solid #1e3a5a;border-radius:8px;padding:16px 18px;vertical-align:top;" |
−
| style="background:#0d1b2a;border:1px solid #1e3a5a;border-radius:8px;padding:12px 16px;font-weight:600;" | 🔷 GROMACS
+
<div style="font-size:18px;margin-bottom:8px;">📁</div>
−
| style="background:#0d1b2a;border:1px solid #1e3a5a;border-radius:8px;padding:12px 16px;font-weight:600;" | 🔷 OpenFOAM
+
<div style="font-weight:700;font-size:14px;margin-bottom:6px;color:#e6edf3;">[[Magus02/File_Transfer|File Transfer]]</div>
 +
<div style="font-size:13px;color:#7a94aa;">scp, rsync, FTP/SFTP, WinSCP, FileZilla.</div>
 +
| style="background:#0d1b2a;border:1px solid #1e3a5a;border-radius:8px;padding:16px 18px;vertical-align:top;" |
 +
<div style="font-size:18px;margin-bottom:8px;">📦</div>
 +
<div style="font-weight:700;font-size:14px;margin-bottom:6px;color:#e6edf3;">[[Magus02/Modules|Environment Modules]]</div>
 +
<div style="font-size:13px;color:#7a94aa;">Loading compilers, MPI, and scientific software.</div>
 +
| style="background:#0d1b2a;border:1px solid #1e3a5a;border-radius:8px;padding:16px 18px;vertical-align:top;" |
 +
<div style="font-size:18px;margin-bottom:8px;">📋</div>
 +
<div style="font-weight:700;font-size:14px;margin-bottom:6px;color:#e6edf3;">[[Magus02/Scheduling_Basics|Scheduling Basics]]</div>
 +
<div style="font-size:13px;color:#7a94aa;">Batch queues, job lifecycle, and scheduling algorithms.</div>
 
|}
 
|}
  
−
<div style="background:#2a1a00;border-left:4px solid #f0c040;border-radius:6px;padding:14px 18px;margin:16px 0;font-size:14px;color:#d4b870;">
+
=== HPC User Guide ===
−
⚠️ <strong>Gaussian users:</strong> Gaussian requires high-memory nodes. Always submit Gaussian jobs to the <code>high_mem</code> queue and request sufficient memory with <code>#SBATCH --mem</code>.
 
−
</div>
 
  
−
== Queue Structure on Magus02 ==
+
{| style="width:100%;border-collapse:separate;border-spacing:10px;"
−
 
 
−
Magus02 uses '''Slurm''' for job scheduling. Jobs are submitted to specific partitions based on required resources, node type, and runtime. Select the appropriate queue to minimise wait times.
 
−
 
 
−
{| class="wikitable sortable" style="width:100%;font-size:13.5px;"
 
−
|-
 
−
! style="background:#0d1b2a;color:#94a8be;" | S.No.
 
−
! style="background:#0d1b2a;color:#94a8be;" | Processor Architecture
 
−
! style="background:#0d1b2a;color:#94a8be;" | Priority
 
−
! style="background:#0d1b2a;color:#94a8be;" | Queue Name
 
−
! style="background:#0d1b2a;color:#94a8be;" | Min Cores / Job
 
−
! style="background:#0d1b2a;color:#94a8be;" | Max Cores / Job
 
−
! style="background:#0d1b2a;color:#94a8be;" | Max Nodes
 
−
! style="background:#0d1b2a;color:#94a8be;" | Wait Time
 
−
! style="background:#0d1b2a;color:#94a8be;" | Max Cores / User
 
−
! style="background:#0d1b2a;color:#94a8be;" | Max Pending Jobs
 
−
! style="background:#0d1b2a;color:#94a8be;" | Max Running Jobs
 
 
|-
 
|-
−
| 1 || AMD EPYC 7763 || 50 || <span style="background:#0d2535;color:#00c9ff;font-family:monospace;font-size:12px;padding:2px 8px;border-radius:4px;border:1px solid #1a4060;">medium</span> || 32 || 128 || 77 || 1 week || 512 || 2 || 2
+
| style="background:#0d1b2a;border:1px solid #1e3a5a;border-radius:8px;padding:16px 18px;vertical-align:top;width:33%;" |
 +
<div style="font-size:18px;margin-bottom:8px;">🔧</div>
 +
<div style="font-weight:700;font-size:14px;margin-bottom:6px;color:#e6edf3;">[[Magus02/Compiler|Compilers]]</div>
 +
<div style="font-size:13px;color:#7a94aa;">GCC, Intel icc/ifort, LLVM/clang, PGI/NVIDIA.</div>
 +
| style="background:#0d1b2a;border:1px solid #1e3a5a;border-radius:8px;padding:16px 18px;vertical-align:top;width:33%;" |
 +
<div style="font-size:18px;margin-bottom:8px;">📤</div>
 +
<div style="font-weight:700;font-size:14px;margin-bottom:6px;color:#e6edf3;">[[Magus02/Batch_Scheduler|Batch Scheduler]]</div>
 +
<div style="font-size:13px;color:#7a94aa;">Job types: serial, OpenMP, MPI, hybrid, array, chain.</div>
 +
| style="background:#0d1b2a;border:1px solid #1e3a5a;border-radius:8px;padding:16px 18px;vertical-align:top;width:33%;" |
 +
<div style="font-size:18px;margin-bottom:8px;">🔀</div>
 +
<div style="font-weight:700;font-size:14px;margin-bottom:6px;color:#e6edf3;">[[Magus02/OpenMP_Usage|OpenMP Usage]]</div>
 +
<div style="font-size:13px;color:#7a94aa;">Compiling and running shared-memory parallel programs.</div>
 
|-
 
|-
−
| 2 || AMD EPYC 7043 || 50 || <span style="background:#0d2a10;color:#39d353;font-family:monospace;font-size:12px;padding:2px 8px;border-radius:4px;border:1px solid #1a5020;">small</span> || 8 || 32 || 16 || 1 month || 128 || 2 || 4
+
| style="background:#0d1b2a;border:1px solid #1e3a5a;border-radius:8px;padding:16px 18px;vertical-align:top;" |
−
|-
+
<div style="font-size:18px;margin-bottom:8px;">🌐</div>
−
| 3 || AMD EPYC 7543 || 60 || <span style="background:#2a2000;color:#f0c040;font-family:monospace;font-size:12px;padding:2px 8px;border-radius:4px;border:1px solid #504000;">high_mem</span> || 32 || 256 || 12 || 2 weeks || 1024 || 2 || 2
+
<div style="font-weight:700;font-size:14px;margin-bottom:6px;color:#e6edf3;">[[Magus02/MPI_Usage|MPI Usage]]</div>
−
|-
+
<div style="font-size:13px;color:#7a94aa;">Open MPI and Intel MPI: compile, run, bind.</div>
−
| 4 || AMD EPYC 7763 || 60 || <span style="background:#2a1010;color:#f85149;font-family:monospace;font-size:12px;padding:2px 8px;border-radius:4px;border:1px solid #502020;">large</span> || 64 || 256 || 24 || 2 weeks || 1024 || 2 || 2
+
| style="background:#0d1b2a;border:1px solid #1e3a5a;border-radius:8px;padding:16px 18px;vertical-align:top;" |
−
|-
+
<div style="font-size:18px;margin-bottom:8px;">🏗️</div>
−
| 5 || Intel Xeon Platinum 8368 || — || <span style="background:#1a0a30;color:#b392f0;font-family:monospace;font-size:12px;padding:2px 8px;border-radius:4px;border:1px solid #3a2060;">GPU</span> || 16 || 16 || 4 || 1 week || 32 || 2 || 2
+
<div style="font-weight:700;font-size:14px;margin-bottom:6px;color:#e6edf3;">[[Magus02/NUMA|NUMA Architecture]]</div>
−
|-
+
<div style="font-size:13px;color:#7a94aa;">Non-uniform memory access, thread pinning, first-touch.</div>
−
| 6 || AMD EPYC 7763 || — || <span style="background:#1a1a1a;color:#8b949e;font-family:monospace;font-size:12px;padding:2px 8px;border-radius:4px;border:1px solid #333;">low / long</span> || 32 || 1024 || All nodes || 1 month || 1024 || 2 || 8
+
| style="background:#0d1b2a;border:1px solid #1e3a5a;border-radius:8px;padding:16px 18px;vertical-align:top;" |
 +
<div style="font-size:18px;margin-bottom:8px;">❓</div>
 +
<div style="font-weight:700;font-size:14px;margin-bottom:6px;color:#e6edf3;">[[Magus02/FAQ_Batch_Jobs|FAQ: Batch Jobs]]</div>
 +
<div style="font-size:13px;color:#7a94aa;">Troubleshooting common Slurm job issues.</div>
 
|}
 
|}
  
−
<div style="background:#0d1a0d;border-left:4px solid #39d353;border-radius:6px;padding:14px 18px;margin:16px 0;font-size:14px;color:#80b880;">
+
== 🔗 Resources & Support ==
−
✅ <strong>Queue selection tip:</strong> For small test jobs or debugging → use <strong>small</strong>. For production multi-node runs → use <strong>medium</strong> or <strong>large</strong>. Memory-intensive applications (Gaussian, VASP single-node) → use <strong>high_mem</strong>. Long-running low-priority jobs → use <strong>low/long</strong>.
+
 
 +
{| style="width:100%;border-collapse:separate;border-spacing:12px;"
 +
|-
 +
| style="background:#0a1a0a;border:1px solid #1a4020;border-radius:8px;padding:16px 20px;vertical-align:top;width:50%;" |
 +
<div style="font-weight:700;font-size:14px;color:#39d353;margin-bottom:8px;">📞 Getting Help</div>
 +
<div style="font-size:13px;color:#7a94aa;line-height:1.8;">
 +
Contact the HPC Administration Team for access requests, technical issues, and software installation.<br><br>
 +
<strong style="color:#e6edf3;">Email:</strong> <code>[email protected]</code><br>
 +
<strong style="color:#e6edf3;">IT Helpdesk:</strong> Submit a ticket via the SNU IT portal
 
</div>
 
</div>
 +
| style="background:#0a1520;border:1px solid #1a3050;border-radius:8px;padding:16px 20px;vertical-align:top;width:50%;" |
 +
<div style="font-weight:700;font-size:14px;color:#00c9ff;margin-bottom:8px;">📄 Downloads</div>
 +
<div style="font-size:13px;color:#7a94aa;line-height:1.8;">
 +
[[Media:Linux_Command_Reference.pdf|⬇ Slurm / Linux Command Reference (PDF)]]<br><br>
 +
All major Slurm commands, job submission flags, and useful Linux utilities in a single printable reference sheet.
 +
</div>
 +
|}
  
−
== Slurm Command Reference ==
+
<div style="background:#2a1a00;border-left:4px solid #f0c040;border-radius:6px;padding:14px 18px;margin:20px 0;font-size:14px;color:#d4b870;">
−
 
+
⚠️ <strong>External Access Reminder:</strong> Accessing Magus02 from outside the SNU campus network requires a VPN connection. Contact the HPC admin team or scan the QR barcode on your welcome email to request VPN access. See [[Magus02/SSH#External_VPN|SSH — External VPN Access]] for details.
−
=== Common Slurm Commands ===
 
−
 
 
−
<pre style="background:#0d1117;border:1px solid #21293a;border-radius:8px;padding:18px;font-family:monospace;font-size:13px;color:#e6edf3;line-height:1.8;">
 
−
# Submit a batch job
 
−
$ sbatch job_script.sh
 
−
 
 
−
# List all running/pending jobs
 
−
$ squeue
 
−
 
 
−
# Check only your jobs
 
−
$ squeue -u $USER
 
−
 
 
−
# Cancel a job by ID
 
−
$ scancel JOB_ID
 
−
 
 
−
# Check detailed job status
 
−
$ scontrol show job JOB_ID
 
−
 
 
−
# View partition / queue information
 
−
$ sinfo
 
−
 
 
−
# Start an interactive session
 
−
$ srun --pty --partition=medium --ntasks=4 /bin/bash
 
−
</pre>
 
−
 
 
−
=== Example Slurm Batch Script ===
 
−
 
 
−
<pre style="background:#0d1117;border:1px solid #21293a;border-radius:8px;padding:18px;font-family:monospace;font-size:13px;color:#e6edf3;line-height:1.8;">
 
−
#!/bin/bash
 
−
#SBATCH --job-name=my_simulation
 
−
#SBATCH --partition=medium
 
−
#SBATCH --nodes=2
 
−
#SBATCH --ntasks-per-node=64
 
−
#SBATCH --mem=256G
 
−
#SBATCH --time=7-00:00:00          # max 1 week
 
−
#SBATCH --output=slurm-%j.out
 
−
#SBATCH --error=slurm-%j.err
 
−
 
 
−
module load intel/oneapi mpi/intelmpi
 
−
 
 
−
mpirun -np $SLURM_NTASKS ./my_program
 
−
</pre>
 
−
 
 
−
=== Slurm Reference PDF ===
 
−
 
 
−
📄 [[Media:Linux_Command_Reference.pdf|Download Linux / Slurm Command Reference (PDF)]]
 
−
 
 
−
<div style="background:#0d1a2a;border-left:4px solid #00c9ff;border-radius:6px;padding:14px 18px;margin:16px 0;font-size:14px;color:#94a8be;">
 
−
For further assistance, contact the HPC Administration Team or raise a support ticket through the SNU IT helpdesk.
 
 
</div>
 
</div>
  
 
----
 
----
−
<div style="font-family:monospace;font-size:11px;color:#484f58;text-align:right;margin-top:8px;">
+
<div style="font-family:monospace;font-size:11px;color:#484f58;text-align:right;margin-top:4px;">
 
Maintained by HPC Administration · Shiv Nadar Institution of Eminence, Delhi NCR · wiki.snu.edu.in
 
Maintained by HPC Administration · Shiv Nadar Institution of Eminence, Delhi NCR · wiki.snu.edu.in
 
</div>
 
</div>

Latest revision as of 06:22, 20 April 2026


Shiv Nadar Institution of Eminence — HPC Infrastructure
Magus02 HPC
Knowledge Base
Your complete reference for accessing, using, and getting the most out of the Magus02 High Performance Computing cluster at SNU. Whether you are a first-time user or an experienced HPC programmer, this wiki has you covered.

HOST: magus02.snu.edu.in ● SYSTEM OPERATIONAL SCHEDULER: Slurm

🚀 Quick Navigation

⚡ New to Magus02? Start here
Everything from account access and SSH login to running your first job — in one guide.
📘 Basics

Getting Started · First steps on Magus02
Linux Shell · Commands and filesystem navigation
SSH Access · Connecting securely to the cluster
File Transfer · scp, rsync, WinSCP, FileZilla
Environment Modules · Loading software on HPC
Scheduling Basics · How the batch system works

⚙️ HPC User Guide

Compilers · GCC, Intel, LLVM, PGI
Batch Scheduler · Submitting and managing jobs
OpenMP Usage · Shared-memory parallelism
MPI Usage · Distributed-memory parallelism
NUMA Architecture · Memory locality and pinning
FAQ: Batch Jobs · Common problems and solutions

🖥️ About Magus02

Magus02 is the primary HPC cluster at Shiv Nadar Institution of Eminence, Delhi NCR, hosted within the Department of Chemical Engineering. It provides researchers, faculty, and students with large-scale parallel computing resources for a wide range of workloads including computational chemistry, fluid dynamics, machine learning, and data analysis.

Cluster at a Glance

TOTAL CORES PEAK PERFORMANCE COMPUTE NODES GPU NODES TOTAL RAM STORAGE
9,064 238 TFlops 57 2 (Tesla A100×4) ~42 TB 750 TB
Node Type Processor RAM / Node Cores / Node GPU Count Total RAM Total Cores
Compute Node AMD EPYC 7763 512 GB 128 — 57 29,184 GB 7,296
High Memory Node AMD EPYC 7543 1,024 GB 64 — 12 12,288 GB 768
GPU Node Intel Xeon Platinum 8368 512 GB 64 Tesla A100 ×4 2 1,024 GB 128
Storage 750 TB Parallel Filesystem

Queue Summary

Queue Node Type Max Cores/Job Max Nodes Wall Time Use Case
medium Compute 128 77 1 week General multi-node jobs
small Compute 32 16 1 month Testing and small runs
high_mem High Memory 256 12 2 weeks Gaussian, large in-memory jobs
large Compute 256 24 2 weeks Large parallel production runs
GPU GPU 16 4 1 week CUDA/GPU workloads
low/long Compute 1024 All 1 month Long, low-priority jobs

→ See Magus02 Cluster Page for full queue details, connection instructions, and application list.

📚 Documentation Sections

Basics (New Users)

🖥️
Access, login, VPN, first steps on the cluster.
⌨️
Essential commands for navigation, files, and utilities.
🔒
Connecting from Linux, macOS, and Windows.
📁
scp, rsync, FTP/SFTP, WinSCP, FileZilla.
📦
Loading compilers, MPI, and scientific software.
📋
Batch queues, job lifecycle, and scheduling algorithms.

HPC User Guide

🔧
GCC, Intel icc/ifort, LLVM/clang, PGI/NVIDIA.
📤
Job types: serial, OpenMP, MPI, hybrid, array, chain.
🔀
Compiling and running shared-memory parallel programs.
🌐
Open MPI and Intel MPI: compile, run, bind.
🏗️
Non-uniform memory access, thread pinning, first-touch.
❓
Troubleshooting common Slurm job issues.

🔗 Resources & Support

📞 Getting Help

Contact the HPC Administration Team for access requests, technical issues, and software installation.

Email: [email protected]
IT Helpdesk: Submit a ticket via the SNU IT portal

📄 Downloads

⬇ Slurm / Linux Command Reference (PDF)

All major Slurm commands, job submission flags, and useful Linux utilities in a single printable reference sheet.

⚠️ External Access Reminder: Accessing Magus02 from outside the SNU campus network requires a VPN connection. Contact the HPC admin team or scan the QR barcode on your welcome email to request VPN access. See SSH — External VPN Access for details.

Maintained by HPC Administration · Shiv Nadar Institution of Eminence, Delhi NCR · wiki.snu.edu.in

Retrieved from "https://wiki.snu.edu.in/index.php?title=Magus02_new&oldid=1175"
No categories assignedEdit