Skip to content

Plan a cluster around the workload

Audience: Both users and administrators · Reviewed: 2026-10-04

Version and teaching scope

Use documentation matching your installed release and local policy. Check sinfo --version; upstream documentation identified itself as 26.05 at review. Paths, partitions, accounts, modules, memory, MPI, GPUs and services are site-specific. Example programs beginning ./ must already exist. These are teaching examples, not a validated deployment recipe or evidence that a cluster was modified.

Responsibilities before hardware

An HPC cluster combines compute, networking, storage, software, identity, and a workload manager. Slurm grants resources and starts work on them; it does not replace the operating system, provision every node, or automatically make a scientific application parallel.

Separate responsibilities before selecting hardware:

  • A login node is for editing, small preparation steps, transfer, and submission. Heavy calculations and large compilations belong in an allocation or a designated build service.
  • Compute nodes run applications. A controller running slurmctld manages scheduling; each compute node runs slurmd, with job-step processes managed by slurmstepd.
  • Accounting records usage and policy associations. slurmdbd brokers access to the accounting database; users should not receive direct database credentials.
  • Storage needs distinct policies for home directories, shared project results, and temporary scratch. Fast scratch is not necessarily backed up.

Match resources to scientific work

For quantum chemistry, describe representative jobs before buying nodes: independent molecular screenings, one-node threaded calculations, memory-heavy correlated methods, tightly coupled MPI jobs, and GPU-enabled codes stress different resources. Requesting more nodes cannot rescue a program that only uses shared memory. License limits and supported parallel launch methods also matter.

A capacity worksheet

Create a capacity worksheet with measured RAM per job, scratch capacity and I/O pattern, useful thread/rank counts, typical wall time, and concurrent job count. Benchmark representative inputs, including convergence and restart behavior. Compare scientific throughput and cost, not just peak FLOPS. A small local VM lab can teach scheduling but cannot establish interconnect or storage performance.

Checkpoint: Explain where a batch script is submitted, where it executes, where its data resides, and which component decides when it starts.

Official sources

Learning path · Next: Build a safe learning lab; provision production deliberately