December 6, 2022·6 min read·engineeringnixlinuxsandboxingbuilds

Nix gives a sandboxed build a full range of user identities

Nix 2.12 can run Linux builds as namespace root with 65,536 UIDs. Automatic identity allocation and cgroups support nested container workloads without assigning the build host's root identity.

A build that starts a container needs more than the ability to execute the container program. Its processes may expect a root identity and multiple service accounts inside the container. A sandbox with only one mapped user cannot supply that whole identity range. Nix 2.12 adds Linux builds with 65,536 available UIDs inside a user namespace, where the build runs as UID 0. The December 6 release notes identify running systemd-nspawn inside a build as the main use case.

The important distinction is between an identity inside the namespace and its identity on the build host. Nix maps the build's namespace identities onto an allocated host range. Seeing UID 0 inside that view does not mean the builder received the host's root UID. The mapping is the mechanism that makes the new build environment meaningful.

I value Nix's ability to describe system artifacts as builds. Being able to exercise a service environment during such a build is useful when its behavior depends on more than one process. The identity and resource boundaries need to be declared as carefully as the software inputs.

The derivation requests a system capability

A derivation opts in by requesting the uid-range system feature. The release notes show the declaration directly.

requiredSystemFeatures = [ "uid-range" ];

This attribute belongs to the derivation's build requirements. It states that the work needs the special environment, rather than relying on the builder to happen to support it. The build's source inputs alone cannot establish whether the executing machine can provide the requested identity arrangement.

The local builder implementation in 2.12 checks that a range-using build has a sufficiently large allocation. It requires at least 65,536 identities and reports that automatic UID allocation must be enabled when that requirement is unmet. It also rejects the feature outside sandboxed builds and on unsupported platforms.

Those checks are more specific than a claim that Nix can now run arbitrary privileged software. The implementation provides a particular sandbox configuration with a particular identity map. A build requiring it must ask for it, and the executing system must satisfy the corresponding settings and kernel operations.

For a distributed build setup, this declaration also makes the capability visible in the derivation. A maintainer can distinguish a source-level failure from a builder that lacks the requested environment. That is useful even before considering the details of the workload running inside it.

Automatic allocation replaces a fixed account list

Nix 2.12 also introduces experimental automatic UID selection for builds. The settings documentation embedded in the source contrasts it with taking users from build-users-group. This removes the need to create a fixed set of nixbld accounts for that mode.

The documented Linux starting point is UID 872415232, also written as hexadecimal 0x34000000. A separate starting point is defined for macOS, but the full namespace-range build described here is Linux-specific. Support for automatic allocation on more than one platform should not be confused with support for every namespace feature on each platform.

The source defines start-id as the first UID and GID for dynamic allocation. It also defines id-count as the size of the identity pool. These settings describe host-side resources available for allocation. The namespace map determines which identities the particular build sees inside its sandbox.

extra-experimental-features = auto-allocate-uids cgroups
auto-allocate-uids = true

This configuration fragment enables the experiments involved in the range-build path. The ordinary sandbox and platform prerequisites still apply. It is a daemon or builder configuration choice, not something a derivation can grant itself merely by requesting a feature.

The separation is useful for shared builders. Project source describes what a build requires. Builder configuration describes which execution facilities its operator enables. Recording both gives a clearer account than hiding a host-specific prerequisite in a shell command deep inside the build.

Mapping connects namespace users to host users

The builder source writes the child's uid_map and gid_map files after creating the user namespace. It obtains the host starting identity from the allocated build user and obtains the range length from that allocation. The namespace-side starting identity comes from the sandbox's selected UID and GID.

The map therefore relates a range in one view to a range in the other. Several distinct accounts can exist inside the build without all collapsing to one host identity. That is the capability a containerized service environment needs when its processes change among service accounts.

The implementation also prepares account files inside the sandbox. Its generated password file supplies entries for root, the Nix build user, and nobody. These files give processes a consistent account view in addition to the kernel's numeric mappings. Account names and mapped numeric identities solve related but different parts of creating that environment.

This arrangement does not make the entire host filesystem available. The builder source describes binding the closure of declared input paths into the build's store view, instead of exposing the whole store. The namespace identity work supplements those existing input boundaries. It does not replace them with an unrestricted view simply because the builder sees itself as root.

My preference is to explain these layers separately. The derivation supplies inputs, the sandbox determines filesystem visibility, and the user map determines how process identities relate to the host. A container workload needs the combined environment, but each mechanism answers a different question.

Cgroups are part of the range-build path

The settings documentation says cgroups are required and enabled automatically for derivations requesting uid-range. The local builder requires the experimental cgroups feature when the allocated build identity has a range or when cgroup use is requested explicitly. It then obtains the cgroup filesystem and the build process's cgroup context.

This means the settings in the earlier configuration are connected. Automatic allocation makes the identity range available. Cgroup support is a requirement of the builder's execution path for that range. Treating only the allocation flag as the whole setup would omit a condition enforced by the implementation.

The release also adds an explicit use-cgroups setting for Linux builds generally. That is broader than the range feature, since ordinary builds can request cgroup execution through configuration too. The same release says nix build --json can report top-level derivation statistics, including CPU information when cgroups are enabled.

The existence of those statistics does not establish a performance improvement for the build. It supplies information about execution. The range feature's main purpose is to permit an execution arrangement that some workloads need, while the statistics provide a way to observe aspects of what ran.

The example boots a service environment inside the build

The systemd-nspawn test in the 2.12 source demonstrates a concrete workload. It constructs a NixOS configuration with an HTTP service and a test service. The test service writes a message, records the device tree, and requests a page from the local HTTP server.

Its build requests uid-range and starts the resulting configuration through systemd-nspawn. The command binds the Nix store read-only, binds the output directory, and uses a private network. It also prepares the cgroup filesystem for the nested environment. These details show that the example exercises a service manager and its surrounding environment, rather than merely launching an ordinary command with another label.

The HTTP request is directed at localhost within that environment. The test can check interaction between its own services without turning its private network into a dependency on a public web endpoint. Its output captures results from inside the containerized system back into the build's output directory. The test service also configures explicit service-manager exit actions, including a failure exit status, so the nested environment has a defined termination path after the service runs.

That is a useful application of declarative system construction. The same expression defines the system to launch and the build that exercises it. The new UID range supplies the account model required by the nested service environment, while explicit mounts determine which inputs and outputs cross its filesystem boundary.

Nix 2.12 consequently makes a specific class of system-level build workload possible through declared requirements. The supported Linux builder maps an allocated host range into namespace identities, supplies the associated cgroup execution path, and runs the container against declared inputs. The example's service interaction then becomes work performed inside that build environment.