Reproducible HPC
Nix, containers, and workflow tools for HPC environments
Context
HPC environments drift. Module systems, shared libraries, and compiler versions change from under you; the calculation that ran last month will not necessarily run today. A README that lists what you installed is not a reproducibility strategy.
Nix for HPC
Nix pins every dependency from compiler to Message Passing Interface (MPI) implementation in a content addressed store, producing bit-for-bit reproducible builds. Benchmarks on the Elja cluster show the resulting binaries match the performance of manually tuned builds (Goswami et al. 2023).
Literate reproducible workflows
Multi-tool pipelines (alignment, tree building, statistical analysis) drift worst of all because each stage is usually someone else’s script. We combined org-mode with Snakemake so the analysis prose, code, and execution live in one file, regenerated deterministically (Goswami and S 2023).
rsx records every analysis argument in a schema-versioned TOML profile and ships a Zenodo archive that regenerates every figure and reported number (Goswami and Goswami 2026). A failed run still keeps the hydrated profile, so the configuration that produced a result is not lost with the process.
Broader ecosystem work
Since 2021 I have consulted for the Icelandic National Competence Center for HPC and AI (IHPC) on reproducible software environments for Elja, including the Nix workflow above. I also maintain conda-forge feedstocks, contribute to pixi and spack, and help projects adopt reproducible builds. The blog collects the Nix, pixi, and packaging patterns.
The questions in this line are how an environment stays reproducible without dropping the native build, how to report time and energy per correct result, and how deployment and recovery behave across sites. Nix, EasyBuild, and EESSI are three ways to close over an environment. Flowy, eOn, and rsx each have a scientific result that can be checked apart from the systems measurement. metatomic is where one learned model runs on CPUs and GPUs inside more than one simulation engine.
Code
- hzArchiso – Custom ArchLinux installation media
Related writing
- HPC series on rgoswami.me (per-project Spack baselines, Spack + PyTorch workflow).
- Local Nix without root (how to bootstrap Nix on a cluster where you do not have admin).
See also: packaging tutorials under Teaching for Nix workshop material.
Open directions
- A “BLAS for chemical kinetics” (from the thesis conclusion): standardized, optimized building blocks that any simulation code can call, the way LAPACK standardized linear algebra.
- Extending the Nix-based HPC approach to GPU-accelerated workflows with proper Compute Unified Device Architecture (CUDA) and ROCm dependency management.