Docker Engine 24 separates image content from container filesystems

An image downloaded from a registry is not yet the filesystem on which a process runs. Docker Engine 24.0, released May 16, introduces experimental containerd content storage into the Engine. This makes the distinction between stored image content and runnable filesystem state more visible in Docker's architecture. The release also removes the deprecated AUFS and legacy overlay drivers, while removing deprecated BuildKit buildinfo support in favor of provenance attestations.

These changes concern different parts of image handling. The experimental backend changes how the Engine manages image content. Driver removal changes which older local storage implementations remain available. Provenance changes how information about construction accompanies a build result. Their common concern is that an image now needs to represent more than a directory tree ready for execution.

I value Docker's content-addressed artifacts because they give a build result an identity that can be checked. The storage backend matters when that identity refers to a collection of related objects. Treating the registry artifact and the mounted root filesystem as separate representations makes those relationships easier to understand.

Stored content precedes runnable snapshots

The containerd 1.7.1 content-flow documentation describes the progression. Image content enters the content store. The snapshotter creates committed snapshots from filesystem layers, then creates an active snapshot above the image's final layer. That active snapshot supplies the root filesystem used to create a container.

The content store keeps the downloaded objects addressed by their hashes. A snapshot is the filesystem representation prepared from those objects. Keeping the two concepts distinct explains why downloading an image and preparing it for execution are separate operations. The first obtains content, while the second makes filesystem state usable by the runtime.

This separation is useful even when the same local disk holds both representations. A compressed layer blob has a distribution identity and representation. A prepared filesystem must provide files and directories to a running process. Storage accounting and lifecycle management need to know which representation an operation consumes. Calling both of them an image obscures that distinction.

It also helps explain the role of a snapshotter. The content store does not itself define how to present all the image's filesystem changes as a container root. The snapshotter supplies that part of the process. Containerd's documentation identifies separate local storage areas for content and snapshots, which makes the division concrete at the filesystem level.

A reference can identify several related objects

The same document describes an index that points to manifests for different platforms. Each manifest then points to configuration and layer content. Platform selection occurs by following the appropriate manifest descriptor. An image reference can therefore identify a collection containing several runnable variants, rather than one filesystem configuration.

The OCI Distribution Specification v1.0.1 describes retrieval of manifests and blobs. A manifest request identifies the repository and a tag or digest. Blob requests use content digests. Clients can retrieve each piece separately, so the artifact's structure is a set of references between independently addressed objects.

That structure affects copying and inspecting an image. A tool that only understands one selected runnable manifest has a narrower view than a tool that preserves the surrounding index. Whether that difference matters depends on what the consumer needs. A machine running one platform and a release process preserving all supported platforms are performing different operations on related content.

The experimental Engine backend should be understood in this context. Its purpose concerns image content management, not a claim that every container suddenly needs a different filesystem arrangement. A conventional single-platform service remains a valid use of container images. The content model has to accommodate more complex artifacts without reducing them to that one case.

Attestations are attached information

BuildKit's attestation storage document at version 0.11.6 explains how additional information can accompany the runnable result. It stores attestation manifests in the root image index. An attestation manifest refers to statements about a platform manifest, and those statements can describe provenance or a software bill of materials.

This is an association with an artifact, not an instruction to put the attestation into the application's filesystem. A consumer examining a build may need it, while the process executing the service may have no use for it. The artifact structure needs to preserve that association without confusing the extra manifest with another runnable platform.

BuildKit handles that distinction by giving the attestation descriptor an unknown operating system and architecture. The storage document shows the following platform value. It is a descriptor fragment, not a complete image index.

{
  "architecture": "unknown",
  "os": "unknown"
}

Annotations identify the descriptor as an attestation manifest and associate it with the digest of the object it describes. The combination lets tooling find the relationship while avoiding ordinary platform selection for the attestation. This is a specific example of why an index may contain objects beyond the platform variants an operator expects to run.

The distinction also matters to tools that copy artifacts. Preserving a runnable image while omitting its associated statements changes the information available to the next consumer. The service may still execute, but a later build investigation has less evidence about how the artifact was produced. Execution success alone does not show that every useful part of the distributed artifact survived.

Provenance describes construction

The versioned BuildKit provenance documentation describes statements in the SLSA provenance format, wrapped as in-toto attestations. BuildKit can record build materials with immutable checksums and identify the frontend used. More detailed output can include the Dockerfile and mappings between source instructions and resulting layers.

This explains the difference between provenance and an image's runtime configuration. A startup command says how a process should begin. A provenance statement describes construction of the artifact. Both refer to the same release result, but they answer different questions during an investigation. Keeping construction information associated with the output makes it available without turning it into runtime configuration.

The documentation distinguishes minimum and maximum detail. Minimum mode excludes build-argument values and secret identities. Maximum mode includes build-argument values and metadata about secret and SSH mounts. That choice affects the information published alongside the artifact. Passing confidential values as ordinary build arguments has consequences beyond whether the final application can read them.

BuildKit generates separate provenance for each platform variant of a multiplatform image. That preserves the relationship between a statement and the particular build result it describes. The documentation also describes local and tar outputs, where provenance can appear as a separate file named provenance.json. Its location consequently depends on the exporter, even when the information concerns the same construction process.

The format alone does not establish that every statement is trustworthy. A consumer still needs a reason to trust the builder that generated the information. Provenance makes the description explicit and machine-readable. It gives a policy or an investigator something concrete to evaluate, without proving that the build environment was operated correctly.

I prefer this association between construction information and a specific artifact digest. A build log found elsewhere can be difficult to connect unambiguously to the result being examined. A statement that identifies its subject makes that connection part of the data model. Its usefulness still depends on preserving the statement and evaluating its source.

Experimental storage and removed drivers have different consequences

The Engine release labels the containerd backend experimental. That is separate from the removal of AUFS and legacy overlay. The latter refers to specific deprecated drivers. It should not be read as removal of overlay2, which the same release notes discuss in a container-startup improvement.

A host depending on a removed driver faces a compatibility change even if its administrator has no interest in the experimental backend. Conversely, exploring the new content store is a deliberate choice about a different implementation. Combining both topics into a single instruction to enable containerd would give the wrong account of what the release requires.

Engine 24 therefore has two immediate audiences in image handling. Operators using the removed drivers must account for that removal. Developers exploring richer image artifacts can examine the experimental content-store implementation and the provenance format that replaces buildinfo. The relevant objects include stored manifests and their associated statements, as well as the snapshots used to execute a container.