(email) |
|||
| Line 31: | Line 31: | ||
tar -czvf ~/my_code_backup_$(date +%F).tar.gz /data/home/$USER/my_project_folder | tar -czvf ~/my_code_backup_$(date +%F).tar.gz /data/home/$USER/my_project_folder | ||
</syntaxhighlight> | </syntaxhighlight> | ||
| + | |||
| + | * <code data-path-to-node="10,0,0" data-index-in-node="0">-c</code>: Create a new archive. | ||
| + | * <code data-path-to-node="10,1,0" data-index-in-node="0">-z</code>: Compress using gzip. | ||
| + | * <code data-path-to-node="10,2,0" data-index-in-node="0">-v</code>: Verbosely list files processed (optional). | ||
| + | * <code data-path-to-node="10,3,0" data-index-in-node="0">-f</code>: Use archive file. | ||
=== Step 3: Synchronize to Backup Directory === | === Step 3: Synchronize to Backup Directory === | ||
| Line 39: | Line 44: | ||
rsync -ahP ~/my_code_backup_*.tar.gz /backup/$USER/ | rsync -ahP ~/my_code_backup_*.tar.gz /backup/$USER/ | ||
</syntaxhighlight> | </syntaxhighlight> | ||
| + | |||
| + | * <code data-path-to-node="18,0,0" data-index-in-node="0">-a</code>: Archive mode (preserves permissions and symbolic links). | ||
| + | * <code data-path-to-node="18,1,0" data-index-in-node="0">-h</code>: Human-readable output. | ||
| + | * <code data-path-to-node="18,2,0" data-index-in-node="0">-P</code>: Shows a progress bar during the transfer. | ||
==== Option B: Copying a Directory Directly ==== | ==== Option B: Copying a Directory Directly ==== | ||
Latest revision as of 11:24, 20 March 2026
HPC Cluster Upgrade: Data Backup Guide (March 2026)
| Maintenance Window | Starting 27th March 2026 |
|---|---|
| Backup Location | /backup/<username>
|
| Storage Limit | 10 GB per user |
| Recommended Data | Source code, scripts, configuration files, and small datasets. |
Overview
During the extensive upgrade planned from 27th March, we aim to keep all data intact. However, as a safety measure against any untoward incidents, all users are requested to back up their critical work from /data/home/<username> to the temporary backup partition.
Step 1: Verify Current Usage
Check the size of your home directory to ensure you stay within the 10 GB limit for the backup partition.
du -sh /data/home/$USER
Step 2: Compress Your Critical Files
To save space and speed up the transfer, we recommend creating a compressed archive (tarball) of your code and scripts.
# Create a compressed backup of a specific project
tar -czvf ~/my_code_backup_$(date +%F).tar.gz /data/home/$USER/my_project_folder
-c: Create a new archive.-z: Compress using gzip.-v: Verbosely list files processed (optional).-f: Use archive file.
Step 3: Synchronize to Backup Directory
Use rsync to copy your files to the dedicated backup path. This tool ensures file integrity during the transfer.
Option A: Copying a Compressed Archive (Recommended)
rsync -ahP ~/my_code_backup_*.tar.gz /backup/$USER/
-a: Archive mode (preserves permissions and symbolic links).-h: Human-readable output.-P: Shows a progress bar during the transfer.
Option B: Copying a Directory Directly
rsync -ahP /data/home/$USER/important_folder /backup/$USER/
Step 4: Verification
Confirm your files are present and check the total space used in the backup directory:
ls -lh /backup/$USER/
du -sh /backup/$USER/
Handling Large Data (>10 GB)
If you have a genuine need to back up more than 10 GB:
- Local Download: Download the data to your local machine using SCP or SFTP.
- Contact IT/HPC Admic: If local download is not feasible, contact the IT team for alternative arrangements. Email: [email protected]
