December 7, 2017·4 min read·engineeringgotestingtoolchain

Go 1.10 beta changes what a repeated test run means

The first beta introduces content-based build decisions and successful-test caching for evaluation. The tagged notes explain the cache conditions and the explicit way to request another execution.

A test command can produce the same successful output either by executing a program again or by reusing a result whose inputs have not changed. Go 1.10 beta 1 introduces the latter possibility into go test. That changes a familiar development habit, so the distinction needs to be visible and the conditions for reuse need to be understandable.

The December 7 announcement asks users to test the beta against their unit tests and workloads. Andrew Bonventre says there are no known problems or regressions at announcement time. That is the starting assessment for prerelease testing, not a claim that testing can no longer find problems.

The notes in the beta tag describe both build caching and test-result caching. They are draft notes for a beta, and the behavior discussed here is the behavior offered for evaluation now. The changes remove some old reasons to install dependencies or force rebuilds, while adding an explicit indication when a successful test result comes from the cache.

Build freshness uses contents

The draft says go build now determines whether packages need rebuilding from source contents, build flags, and metadata in compiled packages. Modification times no longer decide freshness. This matters whenever timestamps are a poor description of what a build actually depends on.

A source file can acquire a different timestamp without different contents. Conversely, changes in build flags may matter even when source modification times remain unchanged. Making the decision depend on contents and relevant build inputs better matches the question the compiler needs answered, whether an existing result represents this build.

The beta notes say the old advice to add -a to force rebuilding when modification times mislead is no longer necessary. They ask users to report cases where the tool fails to detect required rebuilding. That gives compatibility testing a concrete target. A build that previously needed a workaround should now work through the ordinary dependency decision.

The build cache is separate from installed packages in GOROOT/pkg or GOPATH/pkg. The intended benefit includes builds that do not explicitly install packages and work that switches between source copies. Cached build results can remain available even when the current source tree changes and later returns to an earlier state.

I like this direction because a routine build should not require private knowledge of which installation command happened earlier. The tool has the source and build configuration. It should use that information to decide which results remain applicable.

Successful tests have additional inputs

The test cache has a more specific contract. The executable and command line must match a previous run. Files and environment variables consulted by that run must also remain unchanged. When the result qualifies, go test prints the earlier output and replaces the elapsed time with (cached).

Only successful results are reusable. A failing test therefore remains an active failure to investigate on another invocation. The cache also applies only when the command specifies an explicit package list, and only with an allowed subset of test flags. The draft lists flags including -run, -short, and -v among that subset.

Those conditions prevent the broad claim that every repeated go test invocation skips execution. The mode of invocation and the inputs matter. Reading the (cached) marker is the direct way to distinguish a reused success from a fresh execution in the output described by the beta notes.

The documented way to request a new execution is -count=1. A command that names a package pattern can make that policy explicit.

go test -count=1 ./...

That command is useful when the purpose is to observe a fresh run. The cache condition describes tracked files and environment variables, not a general proof that every external system behaves identically. A test whose meaning depends on changing remote state has a reason to request execution even when its local program and tracked inputs are the same.

Testing incorporates selected vet checks

The beta also runs a high-confidence subset of go vet checks before tests. Problems found by those checks act like build errors and prevent execution. This moves some diagnostics into the command developers already use for feedback.

The qualification about a subset is important. The change does not say every available vet analysis becomes a mandatory test prerequisite. It selects checks the tool considers sufficiently reliable for this position in the workflow. The draft documents an option to disable vet, but the normal path now includes the selected checks.

Test output gains a machine-readable path through go test -json and go tool test2json. The notes describe this as support for richer presentations in editors and other tools. A structured event stream lets a tool distinguish test execution events without relying entirely on visual formatting intended for a terminal.

I see the cache marker and structured output as related usability work. Both make the tools decisions observable. A developer needs to know whether a test ran, and an editor needs enough structure to present the result accurately. The beta provides these changes together so existing workflows can be checked against them before the release is final.