Go coverage can now follow an integration test through its executable

An integration test can exercise a complete command successfully while leaving its authors unable to say which application code ran. The test may build an executable, feed it files and inspect its output, yet never invoke the package-level runner that historically collected Go coverage. The March 8 explanation of integration-test coverage describes how Go 1.20 removes that restriction.

The important change is the separation of instrumentation from test execution. go build -cover creates an instrumented application binary. A test driver then runs that binary through its ordinary process interface. Coverage collection no longer requires the driver itself to be a Go test function in the package under examination.

I value this because the behavior worth testing often includes the executable's interface. A command's arguments and input handling are part of its contract. Coverage should help explain those tests without requiring a second representation of the command inside a unit-test runner.

Build once and observe executions

The Go 1.20 release notes introduce the build flag and the GOCOVERDIR environment variable. Instrumentation is selected at compilation. The latter tells the executing program where to write coverage data. Keeping those decisions separate lets a test driver control the executions while a wrapper controls observation.

For a main package in the current directory, a minimal shell workflow has this form. The output directory exists before the executable runs, and the reporting step follows execution.

mkdir covdata
go build -cover -o app .
GOCOVERDIR=covdata ./app
go tool covdata percent -i=covdata

The application-specific test still needs to supply meaningful arguments and verify results. The commands above show the coverage mechanism, not a complete behavioral test. A successful exit and a high percentage do not establish that the application returned the correct data.

The official example makes that distinction visible. It uses mdtool, a demonstration application built around a Markdown library. Its first integration test runs the executable against Markdown files and checks that the process does not crash. That is a limited test claim, even though the executions produce useful coverage information.

In the article's reported run, the initial test covers 48.1 percent of statements in the application package. Adding invocations for two other modes raises coverage to 54.6 percent. Those figures belong to the published example. Their significance is the observable relationship between a changed test driver and the application paths it reaches.

Package selection changes the question

By default, go build -cover instruments packages in the module being built. That default can omit much of the behavior a small executable delegates to dependencies. A command that wraps an external library may have excellent coverage of its own dispatch code while saying little about the library operations it invokes.

The -coverpkg flag selects additional packages for instrumentation. In the article, mdtool delegates Markdown processing to gitlab.com/golang-commonmark/markdown, which lives outside the application's module. Selecting that package makes the resulting report describe both the command and the dependency.

This is a change in the measured set of statements. A percentage has meaning only with that set attached. Coverage for the application's module answers whether the test reaches the application's own implementation. Coverage that includes the Markdown library answers a broader question about execution through the dependency. Neither percentage can substitute for the other without changing the claim.

I prefer an explicit package selection when interpreting integration coverage because the executable boundary and the module boundary need not coincide. A test can drive one process while invoking code maintained in several modules. The instrumentation choice needs to match the part of that execution the report is intended to describe.

The official example reports separate package percentages after expanding selection. That presentation preserves information a single total would obscure. Application dispatch and library parsing can have different gaps, and the next useful input depends on which gap matters to the intended behavior.

Raw data supports several reports

The application emits raw coverage files into GOCOVERDIR. The covdata percent command reads those files directly. For familiar function-level and annotated-source reports, covdata textfmt converts the raw data into the text format already used by go test -coverprofile.

go tool covdata textfmt -i=covdata -o=coverage.txt
go tool cover -func=coverage.txt
go tool cover -html=coverage.txt

This conversion preserves the existing reporting workflow while allowing collection to happen outside the test runner. A developer can inspect functions that remain unexecuted after an integration test. The report then supports a concrete question about missing scenarios, rather than leaving the percentage as the only result.

The article's function report shows that some input paths remain uncovered even after hundreds of file-processing invocations. That illustrates why the number of process executions does not determine the diversity of behavior reached. Repeating one input mode can exercise the same statements many times while leaving another mode untouched.

Multiple processes produce multiple files

Each instrumented execution writes one or more files. An integration suite that starts the application repeatedly therefore accumulates coverage output across those runs. The article describes file growth proportional to the number of executions and notes duplicated content among the resulting files.

go tool covdata merge combines the data into another directory. The same operation can combine results from separate integration runs, and its -pkg option can restrict the selected packages. Collection and aggregation consequently remain separate stages, which is useful when several test drivers exercise the same application.

A merged report states what the combined executions reached. It does not explain which individual scenario reached a statement unless the original collections retain that separation. Keeping per-run data can therefore be useful when the investigation concerns a specific mode, while a merged collection suits a broader coverage report.

For long-running applications, the release notes also introduce runtime/coverage APIs that write data while the process continues. That extends the collection model beyond a command that finishes after one input. The integration-test driver can remain responsible for the behavior it verifies, while the application and coverage tooling provide observations about the code reached during that execution.

This also makes existing process-based tests eligible for coverage collection. Their behavioral checks do not need to move into package tests. A wrapper can select the instrumented build and collect its output while the original driver continues to supply inputs and evaluate application results.