July 21, 2021·4 min read·engineeringterraformjsondrift

Terraform emits resource drift in its JSON planning stream

Terraform 1.0.3 adds resource_drift messages to machine-readable plan output. Integrations can distinguish an observed external change from an action Terraform proposes to perform.

An automation interface needs to distinguish what Terraform discovered from what it intends to change. A resource altered through another tool can produce an observation during refresh and a separate corrective action in the plan. Terraform 1.0.3, released on July 21, adds resource_drift messages to the JSON planning stream so integrations can represent that distinction directly.

The new message concerns changes detected outside Terraform. It complements planned_change, which describes an action Terraform plans to take. Conflating the two can produce a misleading summary. A user interface might describe an external update as a future operation, or fail to explain why Terraform proposes returning an object to its configured state.

I find this useful because Terraform's reusable configuration model also needs a comprehensible operational interface. The configuration alone cannot explain every plan. An integration that exposes observations alongside proposed changes gives the reader the information needed to understand a plan without reconstructing the history from terminal text.

The stream is a sequence of typed messages

The machine-readable UI documentation shipped with 1.0.3 describes one JSON object per line for long-running commands such as plan and apply. A client can process each message as it arrives. The message's type determines how to interpret its other fields.

Every stream begins with a version message containing the Terraform version and the UI schema version. Common fields include a timestamp and a human-readable @message. The presence of that message does not require the consumer to parse English in order to identify a resource or action. Those facts have their own structured fields.

A plan can produce the stream through the documented JSON option:

terraform plan -json

This command's output is a stream of UI events. It is not a single JSON document representing the entire plan. A consumer expecting one top-level object would be applying the wrong data model before it reached the new drift message.

The documentation also allows messages concerning different resources to interleave when Terraform operates concurrently. Integrations need to associate related messages through the resource object. The order in which lines arrive does not imply that all messages about one resource will appear together.

A drift event identifies the changed resource

A resource_drift message includes a change object. That object identifies the resource and gives an action such as update or delete for the detected change. The documented example reports an update and includes a full resource address. The event describes what Terraform discovered about the remote object during planning.

The resource object contains more than a display label. Its addr is the full unique address. Separate fields describe the containing module, resource type, name, and instance key. An instance created by for_each or count can therefore be distinguished from other instances of the same resource declaration without interpreting a prose message.

One field needs particular care. The documentation says implied_provider derives from the resource type and may not reflect the actual configured provider when aliases are involved. A consumer cannot use it as proof of which aliased account or regional configuration performed the operation. The field reports the implication of the type name, with the limitation documented.

The event does not include an attribute-by-attribute explanation of the change. The documentation directs consumers to the JSON plan representation for that detail. A compact progress event can support a list of affected resources, while a detailed review needs a richer representation. Those are different information requirements.

Observation and intention can share an address

The same resource can appear in a drift event and a planned-change event. For example, Terraform may observe that a remote object's value differs from its previous state and then propose an update based on the current configuration. The address connects the observations, but the message types explain their different meanings.

planned_change has a wider set of documented actions, including creation, reading, replacement, and no operation. It can also contain a replacement reason. A drift event should not be interpreted through that entire action vocabulary simply because the embedded object also uses the name change.

An integration can preserve this distinction in its own data model. An observed external deletion is information about what has already happened. A planned creation is an intended response. Counting both as operations Terraform will execute would overstate the plan and conceal its explanation.

I prefer an interface that carries this distinction in data because presentation requirements vary. A terminal summary and a review service can use the same event differently while retaining the meaning of the observation. Neither needs to infer causality from a phrase printed for a human reader.

Consumers must allow the interface to grow

The versioned documentation tells UI clients to handle unexpected message types by displaying at least @message. It also identifies the UI schema as major version zero and experimental at this point. A new event is therefore a practical test of whether consumers can accommodate information they did not previously understand.

The release separately updates automatic provider installation to accept registry entries advertising provider protocol version 6. That change concerns plugin selection, rather than plan interpretation. For planning integrations, the consequential addition is the drift event's distinct meaning. Terraform can now report an external resource change in the structured stream without requiring that observation to be represented as a proposed action.