1.6 Slurm, troubleshooting and reproducibility
The inputs on this page are educational templates that have not been executed for this course. Use an authorized installation, verify version-specific options and establish your own convergence.
1.6.1 Slurm is a site interface
The script has not been submitted. Obtain the actual account, partition, module, executable and MPI syntax from your center. Four ranks and 20 minutes are illustrative small requests, not performance predictions. Some sites use mpirun, wrappers or GPU options; never nest launchers blindly. NSC VASP execution guide; Slurm sbatch.
#!/usr/bin/env bash
#SBATCH --job-name=qcnet-si
#SBATCH --nodes=1
#SBATCH --ntasks=4
#SBATCH --cpus-per-task=1
#SBATCH --time=00:20:00
#SBATCH --output=slurm-%j.out
#SBATCH --error=slurm-%j.err
# Add your site's required account/partition lines before submission.
set -euo pipefail
cd "${SLURM_SUBMIT_DIR:?Submit this script with sbatch}"
# SITE-SPECIFIC: replace the next two lines using center documentation.
module load YOUR_APPROVED_VASP_MODULE
VASP_BIN=vasp_std
export OMP_NUM_THREADS="${SLURM_CPUS_PER_TASK:-1}"
for input in INCAR POSCAR KPOINTS POTCAR; do
test -s "$input" || { echo "Missing/empty input: $input" >&2; exit 2; }
done
command -v "$VASP_BIN" >/dev/null
printf 'Start UTC: '; date -u
printf 'Host: '; hostname
printf 'Job: %s\n' "${SLURM_JOB_ID}"
# SITE-SPECIFIC: srun must match the center's MPI integration.
srun "$VASP_BIN" > vasp.stdout 2> vasp.stderr
printf 'End UTC: '; date -u
Replace placeholders and verify the working directory before sbatch job.sh. A nonzero launch fails this script, but a zero exit remains separate from scientific validation. Inspect scheduler status, stdout/stderr, OUTCAR and SCF/ionic criteria. Two concurrent jobs must not write to the same directory. Slurm srun.
1.6.2 Diagnose by evidence
| Symptom | First evidence | Controlled next step |
|---|---|---|
| Immediate launch failure | Module, executable, allocation, MPI/library errors | Follow site's minimum launcher; keep physics unchanged |
| Species/structure warning | POTCAR order, counts, scale, duplicate atoms | Correct, visualize, start a new directory |
| Every SCF reaches NELM | Iteration trend, geometry, occupations, density | Diagnose instability before extending the cap |
| Symmetry/mesh warning | Vector precision, centering, intended symmetry | Choose a compatible grid rather than suppressing warnings |
| Huge forces or collapse | Units, short contacts, geometry, stress accuracy | Stop and test a fixed-geometry model first |
| Spiky DOS | Mesh, energy grid, integration | Increase relevant sampling; NEDOS alone is insufficient |
| Mislabeled bands | Actual reciprocal basis and path conversion | Rebuild the path for the cell |
| Wall-time termination | Scheduler reason, partial files, quota | Preserve logs and verify compatible restart data |
The table is an order of investigation, not an automatic repair script. More NELM can help a healthy slow sequence, not an unstable one. A memory failure does not mean the scientific cutoff should be lowered; adjust resources or supported parallelization while retaining convergence requirements. NELM; Electronic minimization; KPOINTS symmetry guidance.
1.6.3 Restarts are scientific inputs
Preserve the failed run before changing it. Inspect CONTCAR and verify that CHGCAR/WAVECAR is complete and compatible. ISTART=1 and 2 are not interchangeable: their basis handling differs, which matters for cell changes and Pulay stress. A file's existence is not a reason to choose a restart tag. Fresh static verification using ISTART=0, ICHARG=2 is simpler to audit, though potentially more expensive. ISTART; Volume relaxation.
Archive exact inputs, available build/version information, permitted potential labels/hashes, parent-child dependencies, launch script, actual output with units, convergence studies and analysis code/version. Keep licensed potential contents private. Use this blank manifest until you have actual data:
system: diamond Si, 2-atom primitive cell
method: nonmagnetic PBE PAW
geometry_source: ../04-relax/CONTCAR (verified before copy)
parent_density: ../05-scf/CHGCAR
vasp_version: FILL_FROM_ACTUAL_RUN
paw_dataset: FILL_AUTHORIZED_DATASET_ID
encut_eV: FILL_CONVERGED_VALUE
kmesh: FILL_ACTUAL_GRID
energy_convention: FILL_EXACT_OUTPUT_LABEL
scf_converged: FILL_TRUE_OR_FALSE
ionic_converged: FILL_TRUE_FALSE_OR_NOT_APPLICABLE
execution_status: template_not_run
Update the execution status after an actual run; do not quietly label a plausible template “validated.” Keep calculated and planned evidence distinct. Retain unsuccessful tests that explain your final parameter choice.
1.6.4 Final practical assessment
Give another student permitted inputs, job instructions, convergence protocol and plotting code. Ask them to identify the cell, reproduce the workflow in their authorized installation and explain dataset/version differences. A good report includes limitations, failed tests and precision actually established. By completion you should be able to identify the licensed potential and cell, verify SCF independently of process termination, test the target observable, check force/stress, trace bands/DOS to compatible parents and reproduce choices without redistributing restricted files.