November 27, 2024·6 min read·engineeringterraformephemeralsecretslocking

Terraform 1.10 gives temporary values a separate lifecycle

Terraform 1.10 adds ephemeral values and resources that stay out of saved plans and state. Native S3 locking separately moves coordination into a conditional object write.

A credential used to configure a provider does not necessarily belong in the durable record of infrastructure. Terraform 1.10 introduces a way to express that distinction in the language. Ephemeral values exist during an operation but do not persist in saved plan or state files, and ephemeral resources give providers an explicit lifecycle for obtaining and releasing temporary access.

The 1.10.0 release, dated 27 November, adds those mechanisms alongside native S3 state locking. The documentation shipped with the release labels ephemeral resources as public beta, with workflows subject to change. That status belongs in the adoption decision even though the containing CLI release is a stable version.

I value Terraform's ability to describe reusable infrastructure operations, but persisted credentials are a separate concern from persisted resource identity. This release makes the distinction representable. Its design is most useful when the temporary value's permitted destinations are understood as part of the configuration contract.

Redaction and persistence are different properties

Terraform already supports sensitive values. The 1.10 variable documentation explains that marking an input sensitive suppresses its display in ordinary plan and apply output. Terraform still records sensitive values in state when they participate in persisted values. Redaction therefore does not remove the need to protect state.

An ephemeral variable adds a different restriction. Terraform makes its value available during the current operation while omitting it from state and plan files. A temporary session token can consequently be described as a runtime input without becoming a value Terraform must recover from its saved artifacts.

The two annotations can appear together. In this example, the input is a string whose value is relevant only during the current operation, and ordinary output should treat it as sensitive.

variable "session_token" {
  type      = string
  sensitive = true
  ephemeral = true
}

This declaration states both the display policy and the persistence policy. It does not obtain a token or define how a remote service authorizes it. The execution environment must still supply the value, and any system receiving it retains its own behavior.

Expressions depending on an ephemeral variable become ephemeral too. Assigning the variable to a local does not remove its persistence restriction. This propagation matters because otherwise an ordinary transformation could accidentally make a temporary secret eligible for state storage. A local name changes how the expression is referenced, not the lifetime of its contents.

Temporary values have restricted destinations

Ephemeral values cannot be used everywhere an ordinary value can appear. The resource documentation lists ephemeral contexts, including other ephemeral resources, provider configuration, and provisioner or connection blocks. A configuration that passes an ephemeral result into a context requiring persistence receives an error.

That restriction is central to the guarantee. An ordinary managed resource argument can become part of the state used to manage the remote object. A temporary authentication value supplied to a provider configuration plays a different role. The language needs to prevent the former destination from silently accepting information whose declared lifetime excludes persistence.

The new mechanism is therefore not a general instruction to mark every secret-bearing expression ephemeral. The destination must be compatible with the restricted lifetime. Configuration authors still need to distinguish credentials used to perform an operation from values that define the managed object's durable configuration.

Child modules can pass ephemeral values through explicitly marked outputs. The output documentation at the release tag says these values do not persist after plan or apply. It also states that root-module outputs cannot be ephemeral. A child output can connect runtime values between modules without turning them into ordinary durable outputs.

This makes module boundaries relevant to the lifetime contract. A reusable child module that produces temporary access must expose that result as ephemeral, and callers must use it in a permitted context. The declaration communicates more than the result's type. It also constrains where the result can go.

Providers can open and close access

An ephemeral resource has a distinct lifecycle. Terraform opens it when its result is required, asks the provider to renew it if needed beyond a remote expiration interval, and closes it after dependent providers finish their work for the current phase. The resource documentation uses a leased Vault secret to explain acquisition, renewal and explicit revocation.

Those steps are provider operations, not merely local memory management. A lease remains meaningful in the remote system, so its release needs the provider that understands that system. Closing a supported leased secret can tell the remote system to end the lease. Renewal can extend access while the run still requires it. The core coordinates the resource's lifetime through dependencies, while the provider implements the remote system's semantics.

The release changelog says ephemeral resources are read anew during each evaluation phase. Their results are not preserved in state as durable resource instances. This is why the mechanism fits temporary access obtained for a particular run phase, with the appropriate provider support.

An ephemeral block uses a resource type and local name, and providers document which ephemeral resource types they support. The core language does not make every existing managed resource available as an ephemeral equivalent. A provider must implement the corresponding behavior before its resource type can be used through this lifecycle.

I consider that explicit provider contract a useful constraint. A value that looks temporary to the caller may have renewal or revocation behavior in the remote service. Treating it as a managed resource with an ignored state field would not define those operations. The separate lifecycle gives the provider a place to implement them.

The new ephemeralasnull function provides a further way to handle mixed data. Its versioned reference says it replaces ephemeral contents with null while preserving other contents. This is removal of the temporary data from the result, not declassification of the original secret into an ordinary persistent value.

S3 locking uses conditional creation

State persistence itself gains a separate coordination option. The S3 backend reference at 1.10.0 introduces use_lockfile, labeled experimental and defaulting to false. Enabling it selects S3-native locking for the state.

The implementation in pull request 35661 creates a JSON lock object with a conditional S3 write. Its PutObject request uses IfNoneMatch: "*". Acquisition succeeds only when the target lock object does not already exist, so competing clients do not both establish ownership through an unconditional overwrite.

The lock object contains the operation's lock information. When acquisition fails, the implementation attempts to retrieve the existing information and include it in the lock error. Releasing the lock reads the object, checks its stored identifier against the supplied identifier, and deletes the lock object if the identifiers match.

An illustrative backend configuration can request the new mechanism explicitly. The setting belongs to the backend, independently of any AWS provider configuration used to manage resources in the same module.

terraform {
  backend "s3" {
    bucket       = "example-infrastructure-state"
    key          = "network/terraform.tfstate"
    region       = "us-east-1"
    use_lockfile = true
  }
}

The bucket and key are example destinations. The configuration assumes the storage already exists and that the executing identity has the access required for both state and lock operations. A lock object adds operations beyond simply reading the state object.

Two lock mechanisms can support a transition

The release supports using native S3 locking together with DynamoDB locking. The implementation requires both acquisitions to succeed. It acquires the file lock first, then attempts the DynamoDB lock, releasing the file lock if the second acquisition fails.

That dual behavior supplies a migration property. A client using both mechanisms still coordinates through DynamoDB with clients using the older mechanism. After all participants use the native lock, an operator can consider whether the DynamoDB portion is still needed. Removing the common mechanism while older clients remain would remove their shared coordination point.

Locking and ephemeral values solve different problems around the same execution process. Locking coordinates access to a durable infrastructure record. Ephemeral values prevent selected runtime data from joining that record. Terraform 1.10 makes both behaviors explicit, with opt-in locking and a public-beta lifetime contract for temporary provider resources and values.