March 2, 2022·4 min read·engineeringterraformperformancejson

Terraform 1.1.7 stops repeating work in JSON plans

Reusing one recursive result fixes slow JSON plan generation for deeply nested values and explains why unknown-value metadata matters.

A plan can finish calculating infrastructure changes and still take hours to become useful to another program. Terraform 1.1.7 addresses that problem in JSON rendering of deeply nested objects. The March 2 release notes identify accidentally quadratic work in terraform show -json, with consequences that include timeouts in Terraform Cloud and Terraform Enterprise. The underlying patch removes one repeated recursive call.

The distinction between planning and rendering matters here. A provider need not be waiting on a remote API for a workflow to stall. Terraform also has to translate its internal values into a representation that external tools can consume. This bug arose while describing which parts of a planned value remain unknown.

I care about this boundary because Terraform's reusable definitions are useful beyond a terminal session. A machine-readable plan lets other tools inspect the same proposed changes. Rendering costs that grow badly with object structure can make that integration impractical even when the infrastructure change itself is straightforward.

Why a plan needs an unknown-value description

The JSON-format documentation shipped with this version explains that a plan contains prior state, configuration, and proposed changes. It also includes a partial representation of the expected final values. Some of those values cannot be known until apply, so ordinary JSON values alone cannot convey the whole result.

For an individual change, after contains the known values after the proposed action. The accompanying after_unknown structure describes unknown leaves with boolean true values and omits known leaves. A consumer can combine the structures to understand both the available data and the positions where data is still unavailable. An unknown identifier is different from a known null value, even though a simpler values representation can lose that distinction.

The following fragment illustrates the relationship for a resource with a known name and an identifier that apply will determine. It is a fragment of a change representation, not a complete plan file.

{
  "after": {
    "name": "example"
  },
  "after_unknown": {
    "id": true
  }
}

This separation means a renderer has to traverse nested values and construct the corresponding unknown-value structure. A tool cannot infer all unknown positions merely by searching the known values. Terraform must preserve that information while translating its internal representation into JSON.

The documentation also describes sensitivity metadata separately from ordinary values. Unknown values concern availability during planning. Sensitive values concern disclosure. Those properties have different meanings, so consumers must avoid treating the boolean structures as interchangeable labels for missing data.

The repeated traversal

Pull request 30561 identifies the faulty recursion in unknownAsBool. For a child value, the function first calculated its boolean unknown representation. It then checked whether that result was simply false, which allowed known values to be omitted from the output. When the child contained unknown values, the old code called the recursive function again to store the result.

That second call recomputed a result already available in a local variable. At another level of nesting, the same pattern could repeat. The patch stores the previously calculated result instead. The output requirement stays the same, but Terraform stops doing the duplicate work needed to obtain it.

The author notes that ordinary resource shapes did not make the waste conspicuous. Deeper object structures, including Kubernetes manifests, exposed it severely enough that JSON rendering could take several hours. That report concerns the affected renderer and data shapes. It does not establish that every Terraform plan had become slow, or that every use of a Kubernetes provider encountered the problem.

The patch also adds benchmark cases with increasing object depth. Their generated values include an unknown string alongside known values and then nest the object repeatedly. That construction exercises the branch that previously calculated an unknown subtree twice. The benchmark design follows the structure responsible for the regression instead of relying only on a larger count of otherwise shallow resources.

What the command exports

The versioned show-command reference documents -json as the machine-readable form of a state or plan file. With a plan file, the output includes the plan together with configuration and current state. With no file argument, the command shows the latest state snapshot, so those invocations do not describe the same input.

terraform show -json saved.plan

A saved plan is a binary artifact with more detail than the terminal summary. The JSON-format reference presents JSON export as the interface for external tools because the binary file format is unsuitable for that purpose. The rendering fix therefore affects a documented integration path, including consumers that never display Terraform's own human-readable summary.

I see this patch as a useful example of performance work that preserves meaning. Terraform still has to describe the unknown leaves, and clients still have to interpret them correctly. Reusing the recursive result removes avoidable computation without discarding the information that makes a partially known plan intelligible to another program.