rqlite moves its test jobs to CircleCI containers

A continuous-integration migration exposes assumptions that a familiar build machine has been satisfying implicitly. The compiler may already be installed, a script may find an executable through an environment variable, and a system test may require a Python package that unit tests never touch. Moving rqlite to CircleCI 2.0 made those requirements part of a container-based configuration.

In his 29 August 2018 report, Philip O'Toole says rqlite's continuous testing has moved to CircleCI 2.0. He reports quicker testing and says the earlier migration of hraftd helped, though rqlite required more work. The post provides no timing measurements, so the claim is a maintainer's comparison, not a benchmark.

I like rqlite's small deployment model, but a single daemon binary does not make distributed behavior a single unit-test problem. The migration is useful to examine because its jobs make different kinds of evidence visible in the build configuration.

The work is divided by the behavior under test

The migration pull request replaces the old CircleCI configuration with jobs for ordinary tests, race detection and end-to-end testing. The configuration at the completed migration commit names each job separately in a workflow.

Each job selects a Go 1.10 container image and a working directory corresponding to rqlite's import path. Each checks out the source and obtains dependencies. The environment needed for one job is therefore visible without relying on another job having prepared a shared machine.

The ordinary test job runs vet and the Go tests, with a 60-second test timeout. Race detection runs tests with instrumentation and a 120-second timeout. The separate budgets acknowledge that these executions do different work even when they traverse the same package tree.

This structure also makes a failure easier to locate. A passing ordinary test job does not imply that the race job passed. A system-test failure has its own execution context instead of being hidden behind an index-dependent branch in a common command.

The previous configuration selected work using the CircleCI node index. The new workflow gives the work explicit job names. That changes how a maintainer reads the testing contract, even where the underlying test commands remain familiar.

The daemon test needs its own executable

The end-to-end job adds Python's requests package before invoking the system-test script. It also builds the daemon in its command directory. A Go toolchain alone does not satisfy a test runner written in another language.

The job supplies RQLITED_PATH to identify the resulting executable. The final configuration uses an absolute path beneath the container's home directory. This establishes which daemon the script should start, rather than leaving it to whichever executable a search path happens to find.

The pull request's intermediate changes show repeated attention to that path. The final result is more useful than copying one of the earlier attempts. A migration can have a correct conceptual layout and still fail because a test process looks for its target in the wrong directory.

This matters for a database daemon because the system test exercises a running program. Package tests can call Go code directly. A process-level test must connect its runner to a built executable, start it with suitable arguments and observe behavior through the program's external interfaces.

Those are separate dependencies of the test itself. Declaring them makes the job reproducible enough for another maintainer to understand why it needs Python and a daemon build, rather than assuming that every package in a Go repository has the same execution requirements.

Race instrumentation remains a distinct job

The migration configuration retains GORACE with halt_on_error=1 for the race job. That option asks the detector to stop when it encounters a race. The command and its environment belong together because the latter changes how an instrumented failure is handled.

The pull request also includes synchronization changes in a snapshot test. It adds locks around assignments to connection and metadata state before restoration. This is a concrete example of test code itself participating in the concurrency rules of the component under test.

A database test that modifies internal state can race with background work even if its intended assertions concern persistence. The validity of the fixture matters to the meaning of its result. Instrumenting only production source does not make the test's own shared-state accesses safe.

No new performance claim follows from separating these jobs. O'Toole reports that the container-based service is quicker for this project, but the configuration primarily establishes what executes and where. It does not supply a controlled comparison of wall-clock time or resource consumption.

The migration had an external deadline

O'Toole's earlier account of moving hraftd says support for CircleCI 1.0 was finishing in August. That provides a concrete reason for the migration beyond a preference for a newer configuration format. The testing service itself was changing.

For me, the useful outcome is that rqlite's testing requirements become inspectable as named jobs with declared environments. The system test's executable path and Python dependency are explicit, while race detection retains its own command and budget. That gives future changes to the daemon a clearer specification of the checks its continuous integration actually runs.