March 21, 2018·4 min read·engineeringdockerbuildscompose

Docker separates Dockerfile location from the build context

Docker 18.03 lets a local Dockerfile live outside the build context and adds multiple Compose files for Swarm deployment. A multi-stage build fix strengthens the separation between images sharing a parent.

Keeping a Dockerfile beside deployment tooling should not require sending that whole directory to the builder. Until now, a local Dockerfile supplied by path had to sit inside the build context. Docker 18.03 removes that restriction for local directory builds. The March 21 release notes also include a fix for files leaking between images in multi-stage builds. Both changes concern the boundary between the files a build needs and the files it should be able to affect.

The new Dockerfile handling is a CLI change with a fairly small implementation. Sebastiaan van Stijn's patch in CLI pull request 886 reuses the mechanism already available for a Dockerfile supplied through standard input. The client reads the external file and inserts its contents into the context sent to the daemon. The selected directory remains the source of the ordinary build inputs.

I prefer Docker builds whose inputs are easy to identify. Docker's layered image model is useful partly because it makes the construction of an environment explicit. Allowing the recipe to live elsewhere removes a directory-layout constraint without requiring the input directory to grow around it.

The recipe path does not redefine COPY

Consider a project with its application inputs in context and its Dockerfile in dockerfiles. From the project directory, the new client accepts this command.

docker build --no-cache -f dockerfiles/Dockerfile context

The Dockerfile can copy an input called hello from that context. Its source path still describes a file in the selected input directory.

FROM busybox
COPY hello /hello
RUN cat /hello

The example follows the original patch's demonstration. The file being copied belongs in context/hello, even though the recipe belongs in another directory. Selecting an external recipe does not make the recipe's neighboring files part of the context. That is the useful distinction for a repository which stores shared build definitions separately from application inputs.

Before this change, the client rejected the command during context preparation. The patch includes the old diagnostic stating that the Dockerfile must be within the build context, followed by the successful build using the same layout. Its regression test examines the archive delivered to a fake build API. The archive includes the injected Dockerfile alongside the selected input and .dockerignore.

This implementation avoids requiring a new kind of daemon-side path lookup. The daemon receives the recipe in the build archive, just as it did when the client injected a Dockerfile from standard input. A local path supplied to the CLI remains a client-side concern. The daemon does not need access to the client's project directory structure to find the recipe.

There is a deliberate limit in the patch. The Git-context code retains a check rejecting a Dockerfile path outside that context. The local-directory capability should therefore be described at its actual scope. A successful local build with this layout does not establish that every context source accepts the same path relationship.

Separate writable layers for separate build operations

The multi-stage correction addresses a different boundary. Tõnis Tiigi's Moby pull request 36338 patch changes the builder's handling of image layers. It separates a read-only image layer from the writable layer used for an operation. Copy operations obtain their own writable layer and release it when the operation finishes.

The implementation changes make the lifetime visible. An image mount can create a new writable layer. The copy implementation creates a lazy source for that layer's root and tracks the active layer for cleanup. A destination copy also creates a writable layer, commits its result, and releases the writable layer afterward.

This matters when two build stages share an image parent. Sharing that parent is expected, but it must not imply sharing modifications made while constructing separate descendants. A file introduced for one image should not appear in another image simply because their preceding instructions and base image coincide. The patch adds a multi-stage layer-leak regression test specifically around matching instructions before different copy operations.

I favor multi-stage builds because the final service image can contain only the artifact it needs. That preference depends on stage contents remaining separate. A builder that accidentally carries files between images undermines the meaning of the recipe even when every individual COPY instruction looks correct.

The fix therefore changes more than a cleanup detail. Resource lifetime determines which writable state a later operation can observe. Distinguishing the read-only parent from a newly created writable child gives each operation a clearer ownership relationship. The parent remains reusable while the operation's changes belong to its resulting image.

Stack configuration can have several input files

The release also accepts multiple Compose files when deploying a Swarm stack. Vincent Demeester's CLI pull request 569 patch changes --compose-file from a single string to a collection and adds loader merge behavior. It loads the supplied files before constructing the stack's deployment configuration.

The command can now express a base configuration followed by a deployment-specific file. Repeating the option supplies the loader with the ordered collection of inputs.

docker stack deploy --compose-file compose.yml \
  --compose-file compose.production.yml application

The patch makes the first file's directory the working directory for file-based input. Its loader also rejects mismatched Compose versions between the supplied files. Those implementation details matter when configuration is split across directories or edited by different people. Multiple files are still inputs to one loader, with common rules for paths and schema compatibility.

The initial patch keeps a one-file restriction in the Kubernetes loader. The multiple-file capability discussed here applies to the Swarm deployment path. Treating the stack command as one uniform implementation would hide that distinction and imply broader support than the change provides.

Together, these changes allow more flexible placement of build recipes and deployment configuration while retaining specific rules about the inputs each operation consumes. A local Dockerfile can live outside the application context. Swarm deployment can combine configuration files, and separate build descendants receive separate writable layers. For a repository with shared packaging definitions, the practical improvement is the ability to reorganize those definitions without making unrelated directories part of the application build context.