February 1, 2023·6 min read·engineeringdockerbuildkitbuilds

Docker Engine 23 makes the build graph the default

Engine 23 routes Linux builds through Buildx and BuildKit. The change makes build dependencies, temporary mounts and explicit outputs part of the ordinary Docker workflow.

The command can stay the same while the program doing the work changes. Docker Engine 23.0, released on February 1, makes Buildx and BuildKit the default for Linux builds. An existing script that invokes docker build can therefore enter the newer build system without changing its command line. The release moves a different execution model into the path that ordinary users already take.

BuildKit itself predates this change. Its versioned README records earlier Docker integration and describes a builder organized around a dependency graph. The release is significant because that model no longer depends on a developer remembering to opt in. Defaults determine which behavior a project can reasonably expect when someone follows its build instructions.

I care about this because Dockerfile structure is part of service engineering. A multistage build that ends with a static Go binary in a scratch image is my preferred packaging arrangement. A builder that understands the dependencies between stages gives that structure more meaning than a sequence of intermediate images alone can express.

The client participates in the transition

The tagged Docker CLI implementation shows that this is more than a daemon setting. It forwards build commands to the Buildx plugin. It also handles compatibility decisions before forwarding. That makes the local client installation relevant even when the daemon is available and responds normally.

There are two distinct missing-plugin cases. If BuildKit was not explicitly requested, the CLI can warn and continue with the legacy builder. If the user explicitly enables BuildKit and the plugin is missing or broken, the command fails. Silent fallback would contradict the explicit request. This distinction explains why apparently similar machines can respond differently when their environment variables differ.

The code also preserves the ordinary expectation that an unqualified build creates a local image. Unless a builder name or alias changes the selection, forwarding selects the builder associated with the current Docker context. A user who has explored another Buildx builder should not assume that every invocation has the same output behavior merely because both commands use BuildKit internally.

The release notes identify Buildx as a separate package named docker-buildx-plugin. Packaging is therefore part of the migration. An upgrade procedure that considers only the daemon leaves out a component now involved in the default command. This is especially relevant to scripts that provision build machines from a selected package list.

Dependencies describe the work

BuildKit uses Low-Level Build, or LLB, as its intermediate representation. The versioned README describes a binary format for the dependency graph of build processes. A frontend translates a higher-level build definition into that representation. Dockerfile is one such frontend, which separates the language people edit from the format the solver executes.

That separation is useful when reasoning about a multistage file. A stage can consume files produced by another stage. The dependency is established by that consumption, while unrelated work can remain independent. BuildKit's documentation describes concurrent dependency resolution, so a Dockerfile's textual order is an incomplete description of its execution opportunities.

The engineering consequence is that build structure should express real prerequisites. A packaging stage needs the compiler output it copies. A separate operation that produces unrelated files has no inherent reason to become a prerequisite for that package. Treating the file as a dependency description makes these relationships easier to discuss than treating every earlier instruction as a mandatory step toward every later result.

Concurrency does not make dependencies disappear. A compiler still needs its inputs, and copying an artifact still requires the stage that produces it. The benefit comes from distinguishing necessary ordering from ordering that exists only because a person wrote one section before another. That is a property of the model, without requiring a claim about how much faster any particular project becomes.

Temporary build inputs need explicit treatment

The Dockerfile frontend documentation at the shipped BuildKit tag describes RUN --mount. A build process can use a mounted cache directory or a secret file without treating those inputs as ordinary copied application files. These capabilities already exist in the documented frontend. Making BuildKit the default makes them more accessible to standard Linux build instructions.

A cache mount has a narrower responsibility than the instruction cache. It provides a directory whose contents can survive another builder invocation. The documentation requires the build to work regardless of that directory's contents, including when garbage collection has removed them. A cache is consequently an optimization, and the build must still describe how to obtain the actual result.

For a compiler, this distinction is practical. Reusing earlier compilation work can reduce repeated work after a source change, but an empty cache must remain a valid starting condition. A build that requires an undocumented file to survive in its cache directory has acquired a hidden input. The temporary mount mechanism makes the storage convenient without making that hidden dependency correct.

Secret mounts solve a different problem. A process may need a credential to read a dependency, but the credential is not an application artifact. The frontend documents a secret mount with a required option. An illustrative instruction can require that the named secret exists before running an operation that consumes it.

# syntax=docker/dockerfile:1.3
FROM alpine:3.17
RUN --mount=type=secret,id=build-token,required=true \
    test -s /run/secrets/build-token

The command only checks that the supplied file is nonempty. It does not print its contents or claim to perform authentication. A real dependency-fetching command would consume the file in the same instruction. The example makes the lifecycle explicit without inventing an external service or an authentication protocol.

I prefer an explicit temporary input for that purpose. Copying a credential into an image confuses access needed during construction with files needed by the resulting service. A mount expresses the former directly, while the Dockerfile author remains responsible for what the invoked process writes elsewhere.

The frontend has its own version

The syntax directive in that example selects a Dockerfile frontend image. The frontend documentation explains that external frontends let users share a language implementation independently of the builder's bundled version. This introduces a separate version choice alongside the Engine upgrade, and it deserves to appear in discussions of build reproducibility.

An Engine version identifies a collection of shipped components. A syntax directive identifies the frontend that interprets a Dockerfile. Neither name alone describes every external input the build may fetch. Reproducibility discussions become more precise when they name the component whose behavior is under discussion, instead of using the word Docker for all of them.

The same documentation distinguishes its stable and labs channels. A project that depends on a labs capability makes a different language choice from a project that uses the stable syntax shown above. The default builder transition does not remove the need to state that choice. It does make the frontend directive a more ordinary part of a Linux Dockerfile.

Outputs are another explicit boundary

BuildKit's README describes several output types, including images and local files. It also states that standalone BuildKit keeps results internally unless an output is requested. That detail explains why the solver and the Docker command's user-facing behavior should be discussed separately. A build result exists within the build system before a particular exporter makes it available to another consumer.

A local-directory output can make sense for a build whose result is a test report or a binary. An image output makes sense for a service package. These are different uses of the same construction machinery. Selecting an output is part of defining the deliverable, rather than an incidental action after compilation has completed.

That model also helps explain why Buildx builder selection matters. The compatibility behavior in docker build preserves familiar expectations, while the broader toolkit supports more arrangements. A project should describe the command that produces its intended deliverable, including the destination when necessary. Merely saying that a build succeeded leaves the output question unanswered.

For Linux users, Engine 23 makes these distinctions relevant to the default command. The client chooses how to enter the build system. The frontend describes the operations, while an exporter determines how another program receives the result. A build script consequently belongs beside the Dockerfile when documenting how an artifact is produced. Keeping those responsibilities explicit makes a Dockerfile and its invocation easier to maintain as one build definition.