August 23, 2022·4 min read·engineeringcockroachdbtlsmultitenancy

CockroachDB 22.1.6 binds client certificates to tenant identity

CockroachDB 22.1.6 adds tenant scope to client certificates through URI subject alternative names. Certificate generation and authentication must agree on both user identity and tenant scope.

A username alone does not describe the full identity of a client in a multi-tenant system. The same name can have meaning inside different tenant contexts. A client certificate needs a way to carry that context if authentication is to distinguish access to one tenant from access to another.

The August 23 release notes for CockroachDB 22.1.6 introduce tenant-scoped client certificates. The client name remains in the certificate common name. URI entries in the subject alternative name carry the tenant identity with the username. The documented URI format is crdb://tenant/<tenant_id>/user/<username>.

I like CockroachDB's willingness to make distributed-system boundaries explicit. Tenant identity needs that same treatment in an authentication credential. Routing a connection toward a tenant and proving that the credential is valid there are separate decisions.

The certificate carries a qualified identity

The change adds tenant information to an existing client-certificate mechanism. The URI encodes a tenant identifier and a user name in a structured form. That gives the authentication implementation data it can compare with the tenant context receiving the connection.

For illustration, the documented format can represent tenant 7 and user analyst with a value such as this.

crdb://tenant/7/user/analyst

That string illustrates one certificate field, not a complete certificate or a credential someone can use by itself. The security change requires generation of the field and authentication logic that interprets it. Merely adding an arbitrary URI to unrelated credentials would not establish the behavior implemented by CockroachDB.

The backport in pull request 84371, merged July 26, includes both the certificate-generation command changes and the TLS authentication changes. Its description says one client can have scope for a set of tenant identifiers. The implementation therefore supports more than a single unqualified tenant value attached to every username.

Generation accepts an explicit scope

The patch adds a --tenant-scope flag and stores its values as tenant identifiers. The command-line setter accepts comma-separated values and parses each one through the tenant-identifier parser. It resets the default scope before appending the supplied values, so an explicit scope replaces the default list.

The default is the system tenant. The patch description distinguishes system-tenant scope as the representation used for a global client certificate. That means the default should not be interpreted as automatically producing a narrowly scoped application-tenant credential. The intended scope needs to be part of the certificate-generation decision.

This distinction matters for operational tooling as well as applications. A tool that creates certificates without an explicit tenant list can produce a different scope from a tool deliberately restricted to one tenant. A file named for the same username does not, by itself, explain which tenant access the certificate represents.

I find the explicit scope flag useful because the intended boundary can appear in the operation that creates the credential. The operator can reason about which tenant identifiers belong in it before distributing the resulting certificate and key.

The backport supports tenant diagnostics

The pull request explains a concrete operational motivation. Upcoming serverless deployments need secure access to tenant-scoped diagnostic bundles. Tenant-aware client certificates enable that access without treating every diagnostic client as an unqualified identity across the deployment.

The backport also adjusts the project's multi-tenant test tooling. It detects whether the certificate command supports the tenant-scope flag by inspecting help output. Where available, the tests use tenant-scoped credentials. That makes the authentication change part of exercising multi-tenant access, rather than only changing certificate serialization.

Supporting diagnostics gives the change an operational purpose beyond application login. A support connection also needs a defined tenant context, and its authority depends on the credential it presents. The backport makes that qualified identity available to the tools collecting tenant-specific information.

The release also leaves the distinction between a client certificate and a server certificate intact. Its entry concerns authenticating the connecting client to a tenant. It does not describe changing how that client verifies the server it contacted.

Another authentication change has a different boundary

The same maintenance release permits cookie-based authentication for HTTP endpoints under /api/v2. It requires an encoded session in a cookie named session and the header X-Cockroach-API-Session set to cookie. Without that header, the endpoint ignores the cookie.

That is distinct from tenant-qualified certificate authentication. One change gives an HTTP request an explicit way to present an existing session. The other gives a client certificate tenant context that the TLS authentication path can use. Sharing a security section in the release notes does not make them one interchangeable login mechanism.

For tenant-scoped certificates, the consequential addition is the identity encoded in the signed credential and the matching interpretation during authentication. The username remains recognizable, but its permitted tenant context becomes part of what the certificate expresses. Generation, distribution, and use of that credential need to preserve the same intended scope.