The NixOS minimal image reaches a complete reproducibility check

The useful part of a reproducibility percentage is its denominator. The June 20 announcement points to a result covering every store path in a particular NixOS minimal installation image. A dashboard snapshot from that day reports 1,486 reproducible paths out of 1,486, with none unchecked. It tracks the nixos-unstable iso_minimal job for x86_64-linux.

That is a concrete milestone for a bounded system artifact. It does not establish that every package in Nixpkgs is reproducible, and it does not extend the result to every architecture or installation image. The dashboard identifies the package collection revision used for the build. Its result can therefore be discussed as a test of that input and target rather than as a permanent property of everything called NixOS.

I value declarative systems because the description of a machine can be inspected and rebuilt. This result tests an additional property of that process. Declaring inputs is useful, but the build commands consuming those inputs still need to produce the same bytes if independent reconstruction is to verify an artifact.

A store path identifies inputs without proving deterministic output

The Nix profile manual describes packages living at distinct store locations determined by their build inputs. Different source and dependency choices can consequently coexist. This provides a model for identifying what should be built and for selecting environments without overwriting one package version with another.

The output of a build can still vary for reasons the build recipe has not controlled. The dashboard explicitly warns that nondeterminism may remain dormant until a particular circumstance occurs. A declared source and compiler do not themselves prove that every command in the build behaves deterministically. The reproducibility check examines actual output, which is a stronger observation than merely evaluating the same expression twice.

The distinction also separates reuse from verification. Fetching the same cached binary on two machines can give both machines identical bytes without either rebuilding it. That is useful artifact distribution. Independently rebuilding and comparing output asks whether another execution of the build reaches the same result. June's dashboard describes the latter kind of test.

The paths included in an installation image form a relevant population because the image depends on their availability together. Measuring only the most visible executable would miss differences in a dependency or generated support file. The result covers all paths counted by this image check, making the denominator an important part of the announcement.

The revision and job make the claim repeatable

The snapshot identifies Nixpkgs revision 4b4f4bf2845c6e2cc21cd30f2e297908c67d8611. It also names the release expression and exact attribute for constructing the minimal image. With a checkout already at that revision, the documented target is as follows.

nix-build ./nixos/release-combined.nix -A nixos.iso_minimal.x86_64-linux

The revision is necessary because nixos-unstable changes. Replacing the pinned input with whichever revision a channel selects later would test another system. The attribute is necessary because another image includes another set of packages and build steps. Both names belong to the claim, just as a compiler version belongs to an explanation of a binary.

The snapshot says it was generated on June 20 at 05:47 UTC. That timestamp dates the observation, while the discussion post reports the achievement later that day. Together they provide a public result and the announcement interpreting it. No later dashboard state is needed to explain what was measured.

The announcement attributes reaching the result to a recent staging-next merge. It also says the author obtained a passing result for the 21.05 ISO locally, with a qualification about having performed the check correctly. That second statement is useful additional evidence but has a different scope from the dashboard. It should not be silently treated as the same recorded target or as an independent published result with identical detail.

The tests vary execution conditions

The dashboard says each build runs twice at different times on different hardware with different kernels. Those variations are intended to expose output dependence on the surrounding build environment. A comparison that simply retrieves an existing output again would not exercise the same question. The description establishes that the project is rebuilding under changed conditions.

The method still has stated limits. At this point the project does not inject randomness at the filesystem layer. It names disorderfs, which changes directory-entry ordering, as a possible further step. Filesystem ordering is therefore an explicitly identified dimension not yet deliberately perturbed by the reported process.

This is a useful level of precision for a successful test. The authors do not need to claim exhaustive exploration of every possible environment in order for the comparison to matter. They report the conditions changed and identify an additional condition that could be varied. The result provides evidence under those conditions, with room for a more demanding method.

It would be wrong to reinterpret 100% as mathematical certainty about all future rebuilds. The page itself rejects that inference and compares it with the impossibility of proving a general absence of software bugs through testing. Its purpose is to track results over time and identify specific nondeterminism when it appears. A complete passing population can coexist with uncertainty about conditions not exercised.

I find that qualification strengthens the result because it makes the evidence assessable. A percentage without a method offers little basis for comparison. Here the target and execution differences are available, and the dashboard openly describes a missing source of variation.

A failing comparison can retain both outputs

The snapshot documents a local workflow using the hello package. After a normal build, a check rebuild compares the new output with the existing store output. Keeping the failed result makes the disagreement available for inspection.

nix-build . -A hello
nix-build . -A hello --check --keep-failed

The page illustrates a differing result kept under a .check path. diffoscope can then compare the original and rebuilt outputs. This turns a failed reproducibility check into material for diagnosis. Instead of reporting only that a derivation disagreed, a maintainer can inspect the differences in the resulting files.

The preserved check output has a particular lifecycle. The dashboard warns that it is not a valid store path and the next garbage-collection run will delete it. That matters for a debugging workflow because the output is temporary evidence. A maintainer planning a later comparison must account for its retention rather than assume it has the same protection as an ordinary referenced package.

These commands illustrate the method for a small package. They are not evidence that running them for hello verifies the entire installation image. A package-level failure and the population-level result answer different questions. The former helps diagnose one build, while the latter records coverage of the specified image's paths.

The milestone gives later work a defined starting point

The announcement names a fully reproducible GNOME image as the next goal. That is a separate target with additional packages. Its inclusion as future work reinforces the bounded meaning of the minimal image's success. A desktop image cannot inherit a passing result merely because it shares some of the same system components.

Likewise, a newer revision needs its own result. Adding or changing a package changes what the image asks the build system to produce. Continuous checks are useful because the package collection keeps changing even after one complete passing comparison. The June snapshot supplies an exact reference against which such new observations can be described.

For users, the practical significance is an improved basis for independent reconstruction of a bootable distribution artifact and its dependencies. The test does not audit source code for malicious behaviour or prove that a declared dependency was the right one to select. It checks consistency of output for the specified build. Those other questions require different evidence.

The distinction is particularly important when discussing trust. Reproducible output can help another builder verify that an artifact corresponds to the same build inputs. It cannot establish that the inputs are benign. A malicious program can also build deterministically. The value of the check comes from the particular mismatch it can detect, not from substituting for every other review of software.

June's recorded result is therefore precise enough to be useful without broadening it. The unstable minimal x86_64 image at the named Nixpkgs revision has all 1,486 counted paths reproducing under the dashboard's method. The next work can expand the target population and vary additional conditions while retaining that same discipline about what a passing comparison establishes.