Terraform 1.14.6 makes invalid test providers fail

Terraform 1.14.6, released on February 25, fixes a test result that could mislead automation. Its release notes contain one fix, returning an error when a test provider configuration is invalid. The underlying report shows why this deserves attention beyond the usual patch upgrade. A test run could skip its work because of an invalid configuration and still return a successful process status.

I reach for Terraform by default and write providers as well as configuration. That makes me interested in how the test runner establishes that a test was able to run. A successful command should give me useful evidence about the configuration under examination. If setup fails before the test can reach that configuration, I want the failure to remain visible to the system deciding whether to proceed.

A skipped run inside a successful command

The reporter in issue 38084 supplied a provider block in a test file with an unsupported api_version argument. Its run requested a plan. The command printed a skipped run, reported no passed or failed runs, and exited with status zero. The test had not established that the module worked, but a caller checking only the process status could interpret the result as success.

The report also compared this with an ordinary plan using the same provider configuration. That command identified the unsupported argument. Removing that argument exposed a different error about constructing the provider's client. These are separate failures at different stages. The comparison helped isolate the missing diagnostic to the handling of test-defined provider configuration, without requiring a claim that the provider itself had accepted the invalid argument.

A maintainer confirmed that Terraform decodes this provider configuration during test execution and was omitting errors raised at that point. This explains why the problem could escape a casual reading of the output. The runner knew enough to avoid running the test, but the file-level result did not retain the decoding error that explained the skip.

I would treat this as a defect in the evidence produced by the test runner. The module's behavior was still unknown in that run. The command had encountered invalid test setup before it could examine that behavior. A successful exit blurred those two states for an automated caller.

The fix connects the diagnostic to the file

The patch in pull request 38140 changes the provider configuration node in the module test graph. That node calls hcldec.Decode with the configuration body and its specification. Before the fix, the code checked the decoding diagnostics for errors and set the provider status to an error. It then returned, but had not appended those decoding diagnostics to the test file's diagnostics.

After the correction, the code appends the diagnostics returned by decoding before appending the accumulated diagnostics to the file. The existing error check remains. This gives the file result the missing information while preserving the decision to stop configuring the invalid provider. The change does not make the unsupported argument valid or require the test to continue after configuration decoding fails.

That separation is useful when reasoning about a test framework. The provider status controls whether subsequent work can proceed. File diagnostics explain why the file did not complete successfully. Updating one without the other can produce internally consistent scheduling and an externally misleading result. Both consumers need access to the same failure.

The fix is small because Terraform already had most of the necessary behavior. It could detect the decoding problem, stop work that depended on the provider, and represent file diagnostics. The missing operation was the transfer of the decoding diagnostics into that representation. This is the kind of patch I want to understand before attributing a surprising test result to the module under test.

The regression test checks the process contract

The new regression test creates a fixture with an unsupported argument in a test provider block. It initializes the fixture and runs the test command with color disabled. Its assertions check the unsupported-argument diagnostic and the file failure. They also require a nonzero process exit status. Those checks address both the person reading the output and the automation consuming the result.

The expected output still includes a skipped run. This is a detail worth preserving when checking an upgrade. The repair does not require every setup error to appear as a failed individual run. The test file fails because it contains an error, and the process reports failure even though the dependent run could not execute. Looking only for a changed run label would miss the intended behavior.

For a module test suite, I would first upgrade the Terraform executable and rerun the existing command:

terraform test

If that exposes an invalid test provider block, I would correct the block against the provider configuration it expects. I would also check whether the suite now executes the run that was previously skipped. A nonzero exit establishes that the runner reported the setup error. A subsequent successful run needs to demonstrate that the intended test actually executed with valid setup.