Multi-stage builds separate build tools from shipped files

The tools required to produce a program often exceed the files required to run it. A compiler belongs in the build environment, but that does not make it a dependency of the resulting executable. Docker 17.05 gives the Dockerfile a direct way to represent this distinction. Multiple stages can prepare outputs, and a later stage can select the files it needs from an earlier stage.

The 17.05.0-ce release notes date the release to 4 May. They list multi-stage builds alongside build targets and support for build arguments in FROM. These changes concern the structure of a build description. They let one file describe more than a sequence that continually adds to the same final environment.

I prefer shipping a small runtime image containing the program and its actual dependencies. Docker is useful when the build description makes those dependencies legible. Multi-stage builds give the Dockerfile an explicit place to separate production tools from production output without relying on a person to remember which intermediate files to transfer.

A new stage starts with its own base

The Dockerfile reference at the release tag defines FROM as the start of a build stage. It now accepts an optional name through AS. Each new FROM clears the state created by preceding instructions, and subsequent instructions operate on the new base.

That reset is the mechanism that makes selective transfer useful. A later stage does not inherit everything installed in the earlier stage merely because they appear in one Dockerfile. It begins from its own base image. Files cross between stages only through the operations that request them, such as a cross-stage copy.

The same reference extends COPY with --from. Its source can name a previous stage or identify that stage by its numeric index. The source location changes from the client-supplied build context to the filesystem produced by the selected stage. The destination still belongs to the stage currently being built.

A small example can show the relationship without involving a compiler. This creates a text artifact in one stage and copies only that artifact into another:

FROM busybox:1.26 AS prepare
RUN printf 'prepared artifact\n' > /artifact.txt

FROM busybox:1.26
COPY --from=prepare /artifact.txt /message.txt
CMD ["cat", "/message.txt"]

The two stages deliberately use the same base to keep the example focused on the transfer. A real build can use different bases. The important point is that the second stage gets a selected file through COPY, rather than inheriting the complete filesystem changes made by prepare. The runtime command reads the copied output.

The build context and stage filesystem are different inputs

Ordinary COPY takes files from the build context sent by the client. The versioned documentation explicitly prohibits copying a parent directory outside that context. That restriction remains relevant to source files entering the first stage, even though the build can now obtain later inputs from an earlier stage.

Cross-stage copying changes which filesystem supplies the source. A path naming a build output belongs to the producing stage, which is why the example uses an absolute path for the artifact. The client machine does not need to extract that artifact into a second context before the final stage can use it.

This gives the dependency a place in the Dockerfile. The final stage names both the producing stage and the output path. Someone reviewing the file can see the intended transfer directly. If the producer changes its output location, the corresponding copy has a concrete dependency that must change with it.

The source lookup has another detail worth understanding. The reference says that if Docker cannot find a stage with the requested name, it tries an image with that name instead. A name therefore affects how Docker resolves the dependency. Naming stages deliberately makes the intended relationship easier to review, especially in a file containing several intermediate environments.

The runtime still needs its dependencies

Selective copying does not establish that a program is self-contained. The final stage must contain whatever the copied executable needs at runtime. Multi-stage syntax makes it possible to choose those files, but it does not derive a complete runtime environment from an arbitrary compiler output.

This is why the choice of final base remains substantive. A stage that produces an artifact and a stage that runs it answer different questions. The producer needs its toolchain and source inputs. The consumer needs an executable environment appropriate to the resulting files. Choosing a minimal base is useful when that environment actually satisfies the program's requirements.

The Dockerfile reference also distinguishes execution configuration from build execution. RUN executes during image construction and commits the result. CMD supplies defaults for a later container and does not execute during the build. Copying an artifact into the final stage therefore does not by itself check the behavior of the runtime command.

I like that the new structure makes this distinction visible. A file can show exactly where a build output enters the runtime image. It still requires the author to state the intended runtime command and select the runtime dependencies. That is a useful allocation of responsibility because the build description stays explicit about which decisions Docker is carrying out.

A target names the output being requested

The release notes add an option for selecting a build target. Naming stages therefore supports more than a later copy instruction. It gives the build command a way to request a particular stage as its output. One Dockerfile can describe intermediate environments that are useful independently of the final runtime stage.

This matters when diagnosing a build. The environment that contains the tools can be useful even though it is inappropriate as the deployment artifact. Keeping its name in the same file connects it to the runtime image's construction. The intermediate environment does not have to become the shipped image simply to remain accessible as a build output.

Stage names also reduce dependence on numeric positions. The reference accepts numeric indexes for copies, but a meaningful name communicates the dependency when another stage is inserted into the file. The name identifies the intended producer without requiring the reader to count FROM instructions to understand which filesystem supplies a copied file.

The amount of data copied into the final stage depends on the file selection. Each base image also contributes its own contents. Multi-stage builds give authors a mechanism for excluding build-only material, and the resulting image reflects how they use that mechanism.

The reference's existing cache rules also continue to matter. It says that a cached RUN instruction does not become invalid simply because the same command might obtain newer results if executed again. Its package-upgrade example explains that the cached result can be reused on a subsequent build. A named stage makes the producing environment identifiable, but does not change that relationship between an instruction and its cached result. The file still needs an ordering that reflects which inputs should invalidate which work. This connects multi-stage construction to Docker's existing build model instead of treating the new syntax as a separate system that rebuilds every intermediate environment unconditionally.

Base image arguments are another explicit input

Docker 17.05 also allows build arguments in FROM. The versioned reference says the variables must be declared through ARG instructions before the first FROM. Its example uses one argument in multiple base image references. The choice of base can therefore become an input to the build without requiring a rewritten Dockerfile for every value.

That parameterization is separate from stage naming. An argument selects part of an image reference, while a stage name identifies an intermediate build result. Confusing them would make the file harder to interpret because they participate at different points in the build. One determines an initial environment and the other gives a produced environment an identity inside the file.

Defaults also matter. The reference says an omitted tag or digest implies the latest tag. A multi-stage file with several bases consequently has several image selections to account for. Putting the whole construction in one file makes them visible together, but does not make an implicit tag resolve to permanently identical contents.

The useful result of this release is a more complete construction description. Source inputs enter a build stage, that stage produces named outputs, and another stage selects the files it will ship. The final image can omit the producing environment because its filesystem starts from a separate base. Docker now expresses that transfer directly in the same file that describes the build.