February 12, 2020·4 min read·engineeringdockerbuildkituserns

Docker 19.03.6 repairs the assumptions behind image builds

Directory ownership under user namespaces and Windows sandbox expansion explain why a small Engine patch can decide whether an unchanged Dockerfile builds.

A Dockerfile can remain unchanged while a build starts failing because the builder interprets the surrounding environment incorrectly. Docker Engine 19.03.6, released on 12 February, contains examples involving user namespaces and Windows sandbox storage. The release notes also list BuildKit SSH-signature compatibility and clearing triggered ONBUILD rules. These are corrections to existing behavior, with consequences for whether an established build description works under a particular host configuration.

I regard the repeatable build description as one of Docker's most useful contributions. That puts a high value on patches that preserve its meaning across environments. Repeatability needs the implementation to handle the host's identity and storage rules correctly. A build file cannot repair a mistake in the builder's interpretation of those rules simply by being declarative.

The directory and its contents had different owners

The report in Moby issue 40335 describes an interaction between BuildKit and a daemon configured with user-namespace remapping. On Engine 19.03.5, copying into a directory that did not yet exist created that directory with the wrong owner. The copied file had the expected ownership, which made the failure more specific than a general inability to copy files.

The reporter supplied this Dockerfile as a reproduction. It is an illustration of the reported defect, using the same unpinned base-image reference as the issue, rather than a suggested reproducible application build.

FROM alpine
COPY Dockerfile /root/foo/
RUN touch /root/bar
RUN stat -c '%n %u/%g' /root/foo /root/foo/Dockerfile /root/bar
RUN touch /root/foo/bar

The recorded output showed /root/foo owned by UID and GID 65534, while /root/foo/Dockerfile and /root/bar showed zero for both. Creating a file directly under /root succeeded. Creating one under the new directory failed with a permission error. The report says the problem did not occur with either BuildKit or namespace remapping disabled.

That combination is the important diagnostic detail. The application did not suddenly require broader write permissions everywhere. A directory created as part of the copy operation had a different identity from the file beneath it. Looking only at the successful COPY step, or only at the file's owner, would miss the condition that caused the next RUN instruction to fail.

The 19.03 backport identifies the BuildKit fix for that issue. It also records a compilation problem on Windows encountered while integrating the dependency update. Even a Linux ownership repair has to enter a codebase that supports other platforms. That gives a concrete reason for maintaining a distinction between the upstream correction and the Engine patch that delivers it.

I prefer this kind of small reproduction to a workaround that changes application permissions without explaining the discrepancy. It identifies the operation that creates the bad state. The resulting failure then has a specific cause that can be checked against the builder repair.

Windows builds depended on sandbox expansion

The Windows correction has a separate cause. The hcsshim backport includes a workaround for a Windows Host Compute Service problem affecting sandbox expansion on Windows 1903. Its description links failures involving sandboxes larger than 20 GB and builds using a storage-size option. The backport explicitly describes the change as a temporary workaround for a platform bug.

That account is narrower and more useful than saying Windows builds were broken. It identifies the host release, the storage operation and the dependency through which Docker accommodates the problem. An error encountered during an image build does not necessarily originate in the build instructions. Here, the storage used to execute those instructions was part of the failure.

The temporary nature of the workaround also explains why a dependency change can be the appropriate Engine fix. Docker depends on platform behavior that it does not implement itself. Accommodating a known defect in that behavior can restore the existing build workflow while keeping the workaround near the integration code. The release note alone gives little indication of that maintenance decision.

Compatibility includes build-time services

The SSH-forwarding entry in the release notes concerns support for modern signing hashes. The linked BuildKit update entered the 19.03 branch in January. Together with the directory fix, it shows that build compatibility includes operations performed on behalf of an instruction, as well as parsing the instruction itself. Authentication can fail even when the build description has not changed.

The patch therefore matters to builders using these particular configurations without implying that every Docker build gains new behavior. User-namespace remapping exposes an ownership error. Windows sandbox expansion exposes a platform integration problem. SSH forwarding depends on signing compatibility. These distinctions also help determine which component needs attention when the error appears inside a build log. A permission message and a storage failure can interrupt the same workflow while requiring unrelated changes in the implementation. Each failure requires its own explanation, and Engine 19.03.6 supplies repairs within the existing release line.