Docker stack deployment can reject missing configuration variables

A deployment variable can be absent because an operator forgot it, or empty because that is the intended value. Those are different conditions, and a configuration loader needs a way to express which one is acceptable. Docker 18.04 adds mandatory-variable support to stack deployment. Its April 10 release notes include the change, whose implementation distinguishes an unset variable from a variable set to the empty string.

The distinction appears directly in CLI pull request 893. An expression using :? fails when the variable is unset or empty. An expression using ? fails only when it is unset. The text following the operator becomes the diagnostic. That lets the configuration state which inputs must be supplied before the loader can produce a usable stack definition.

I value a repeatable deployment description for the same reason I value a repeatable Dockerfile. The file should contain the decisions needed to interpret it. A required environment value belongs in that description when an empty substitution would change the application's intended configuration.

Presence and content are separate requirements

A stack file can now make its image reference depend on a nonempty release variable. This example uses the new required-value form within the image field.

version: "3.6"
services:
  service:
    image: "registry.example.com/service:${RELEASE:?Set RELEASE}"

If the environment lacks RELEASE, or defines it as an empty string, interpolation produces an error. A nonempty value can pass this particular requirement. The expression does not verify that the image exists or that its contents are appropriate. It establishes one narrower fact before deployment can proceed, that the configuration contains a supplied release value.

The form without the colon makes a different promise. ${SETTING?Set SETTING} requires the environment mapping to contain the name but accepts an empty value. That can be useful when an empty value has a defined meaning in application configuration. Choosing between the forms makes that interpretation visible in the stack file instead of leaving it implicit in a wrapper script.

The patch's tests cover both absent and empty cases. They also check successful substitutions with populated variables and the acceptance of an empty value by the presence-only form. This is a useful test boundary because a single successful deployment example cannot show whether the loader distinguishes those two error conditions correctly.

The implementation extends the interpolation machinery that already understands variable substitutions and defaults. It introduces required-value handling into that machinery, so the error arises while the CLI interprets the configuration. An operator receives the message written next to the variable reference, where the configuration author can explain what input is needed.

This does not turn interpolation into a complete application configuration validator. A populated string can still be wrong for its destination. The practical benefit is earlier rejection of a missing input, with a diagnostic that describes the requirement. A separate shell preflight no longer needs to be the only place where that particular requirement is recorded.

Rotated JSON logs can be compressed

The other operational change is compression for the JSON file logging driver. The Moby patch in pull request 29932 adds a boolean compress option and gzip handling to the log file implementation. Compression is disabled by default. The driver also checks that rotation is configured in a form that can make use of it.

Specifically, the new code rejects compression when max-file is one or when max-size is unset. Compression applies to rotated logs, so a configuration that keeps only the active file does not provide the retained rotated files this mechanism needs. The patch's validation makes that relationship an explicit configuration rule.

An illustrative daemon configuration includes both retention settings with compression enabled. The numeric values here are examples, not a recommendation about a particular application's log volume.

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3",
    "compress": "true"
  }
}

The patch also supports reading compressed files through Docker's logging implementation. Its tests examine compressed rotated output as well as the current file. A reader therefore needs to account for both compressed historical files and the active uncompressed output. This matters because reducing the storage used by older logs would be less useful if it broke the ordinary way of retrieving retained messages.

Compression introduces work when retained files are encoded or decoded. The change provides no universal storage-saving ratio or CPU cost for an application's output, so it should not be described with one. Its concrete capability is keeping rotated JSON log files in gzip form while retaining Docker's ability to read them.

I find these options useful because they put small operational policies alongside the configuration they govern. Required interpolation specifies what input the stack loader must receive. Log compression works with the rotation settings that determine which files remain available. An operator reading the configuration can identify both the condition that stops deployment and the retention arrangement that permits compressed historical logs.