June 24, 2024·6 min read·engineeringdockercontainerdimagesmultiplatform

Docker Engine 27 selects individual platforms when pushing images

Engine 27.0.1 adds platform selection for pushes through the containerd image store. A selected manifest and its parent image index describe different published artifacts.

Knowing that an image supports several platforms does not mean a local image store contains every platform's data. An index can describe manifests whose content is absent locally. A push operation then needs to decide whether it can publish the complete index or should publish one available manifest. That decision changes the object the destination tag identifies.

Docker Engine 27.0.1, released on June 24, adds platform selection to image push when using the containerd image store. It also improves behavior when some content of a multi-platform image is missing. The feature is useful because the local name alone does not fully describe what the engine can upload.

I value Docker's image format because it gives a transferable build artifact an inspectable identity. Platform selection makes the identity of the published object particularly important. A push that succeeds by publishing one platform has accomplished a different task from publishing the original index.

An index and a manifest have different jobs

The engine implementation in PR #47679 distinguishes the image's target descriptor from the individual platform manifests reachable through it. Its helper code can walk manifests whose descriptors are known even when all their content is not present in the local store.

The distinction explains the failure case. The engine can know about a platform from an index, yet lack content needed to push that platform. Merely finding the index does not establish that its entire referenced image can be sent to the destination repository.

The change adds a selected platform to the push operation. When supplied, the engine resolves a matching platform manifest and uses that descriptor as the push target. The selected artifact is therefore a platform-specific manifest, not a rewritten index that retains every relationship in the original object.

This is a semantic choice in the API. The caller asks for a particular platform and receives publication of that platform's image. Other manifests attached through the parent index are outside the selected object. The command should consequently be read as choosing the artifact to publish, rather than as selecting a preferred platform for an otherwise unchanged upload.

The CLI exposes the selected artifact

PR #4984 adds the --platform option to image push. Its help describes publishing a platform-specific manifest as a single-platform image. An explicit operating system and architecture can be supplied with the tag being pushed:

docker image push --platform linux/amd64 registry.example.com/tools/service:test

This example assumes the named image and matching platform content are available in the containerd image store. It does not build that platform or fetch a missing one. The new flag supplies a selection to the engine's existing push operation.

The CLI change annotates the flag with API version 1.46. The client also checks that version before sending a platform option. On the server, the route rejects platform selection for older API versions. Those checks prevent a newer client from assuming that an older endpoint applies the requested selection.

The API represents the platform as a JSON-encoded OCI platform query parameter. Its decoder requires the operating system and architecture. Optional information alone, such as a variant without those identifying fields, is invalid. This is more specific than passing an unvalidated architecture string through to the storage implementation.

Missing content changes the default push path

Without an explicit platform, the engine first attempts the image's existing target. If that target is an index and pushing fails because content is missing, the new implementation can look for a platform-specific candidate. It retries only when the selected target differs from the original descriptor.

The implementation's tests describe how candidate selection should behave. When all platforms are present and no platform is requested, it selects the whole index. When only one is available, it selects that manifest. Where several candidates are available, the daemon's platform can disambiguate the choice if it is among them.

If several candidates remain and the daemon's platform is unavailable, the tests expect ambiguity instead of an arbitrary choice. Explicit requests also fail when the requested platform is absent. These cases preserve the difference between selecting an available artifact and guessing what the caller intended.

The tests also distinguish ARM variants. A request for a variant that the index does not contain is separate from a request for one the index describes but the local store lacks. Both must fail to supply that requested artifact. An available related architecture or variant does not establish that the requested manifest itself is available for publication.

That default behavior helps explain a successful push from a store that contains only part of a multi-platform image. Success can mean that the available single-platform manifest was published. It does not establish that every platform described by the original index reached the destination.

The CLI receives auxiliary progress information for this case and prints a note showing the original index digest and selected manifest digest. Another message describes missing content when the operation cannot complete. The output identifies both the reduced publication and the condition that caused it.

Attestations belong to the publication decision

The CLI change explicitly warns that selecting one manifest excludes other components attached to the multi-platform index, including BuildKit attestations. That follows from what the operation publishes. Selecting a child manifest does not preserve every object associated with its parent.

This matters for a consumer that relies on more than the runnable image. A platform-specific manifest can identify the executable artifact while the original index also connects it with additional metadata. Publishing only the manifest changes which of those relationships the destination tag makes available.

My preference is to describe that result explicitly in release automation. A field called platform can look like a compatibility preference, but here it changes the published object. Any process that records the digest or expects attached provenance needs to use the digest and contents of the object actually pushed.

The engine's progress reporting changes support that interpretation. The patch updates the final digest and size message to use the selected target. Reporting the original index's identity after publishing a child would misdescribe the operation's result. The selected descriptor therefore needs to remain visible through completion reporting as well as content transfer.

There is no need to treat a reduced publication as inherently incorrect. A caller may intentionally want a single platform. The relevant question is whether that is the requested artifact and whether consumers expect the index's additional components. The new option makes the deliberate case expressible, while the fallback note makes an automatic reduction observable.

Runtime information becomes inspectable through the API

The same release exposes more information about the engine's containerd connection. PR #47239 adds a Containerd field to GET /info for API 1.46. It contains the socket address and the containerd namespaces used for containers and plugins.

The source describes that information as intended for debugging. The namespaces belong to Docker's use of the containerd instance, and the API documentation cautions against treating them as shared storage for another system. Discovering the names does not grant another manager ownership of the objects stored there.

This helps separate two questions that can otherwise become confused during investigation. The image store determines which descriptors and content the push implementation uses. The connection information identifies the containerd instance and namespaces used by the daemon. Reading the latter can establish which instance is involved without assuming every containerd client sees the same namespace.

The change also records an image reference on containerd container metadata when the containerd image store is in use. The patch describes this as metadata for lower-level consumers. Its test starts a container and checks that the reference is present through the containerd connection reported by the engine.

The first release is numbered 27.0.1

The release notes explain why the series starts with a patch-looking number. A pre-release mistake created the 27.0.0 tag, and the Go module mirror had already observed it. Reusing that tag would not provide a clean version history, so 27.0.1 is the first release of the series.

The operationally relevant addition remains the relationship between local content and the selected publication. An index can describe more than the engine has locally, and an explicit platform selects a child manifest as the artifact to upload. The API checks the selection, the engine chooses the matching descriptor, and the CLI reports when the published result contains one platform instead of the original index.