January 26, 2017·4 min read·engineeringcockroachdbtlsreplicationoperations

CockroachDB sets a TLS 1.2 minimum and uses locality for replica placement

The January beta requires TLS 1.2 in secure mode and makes replica placement sensitive to declared locality. Placement preferences and mandatory constraints have different failure behavior.

A distributed database needs to know which connections it will accept and where its replicas should live. CockroachDB's January 26 beta changes both decisions. Secure-mode communication now requires TLS 1.2, and replica placement uses node locality during initial allocation and rebalancing. These are configuration changes with direct consequences for connectivity and failure placement.

The contemporary release notes mark the TLS minimum as backwards-incompatible. An older TLS implementation cannot continue negotiating a secure connection simply because its SQL remains compatible. The locality changes separately require an accurate description of the deployment for the allocator to use.

I accept the latency cost of distributed coordination when the consistency it provides is clear. Replica location belongs in the same architectural discussion. The system cannot make useful placement decisions about regions or racks if the deployment never tells it which nodes share them.

TLS policy is enforced by the server

The TLS implementation change, merged January 13, says the client already requires TLS 1.2. This change applies the requirement to the server as well. The server can no longer accept a lower protocol version merely because a connecting implementation is willing to use it.

That distinction matters in a system with more than one possible client. A restriction imposed by CockroachDB's own client does not establish the server's policy for other clients. Requiring the protocol version at the accepting endpoint makes the minimum apply to secure-mode communication regardless of which client initiates it.

The upgrade implication is concrete. A compatible SQL driver still needs a TLS implementation capable of negotiating the server's minimum. Failures at that layer happen before ordinary SQL execution, so changing a query cannot resolve them. The backwards-compatibility note identifies the transport requirement independently of the SQL language changes in the beta. Client deployment configuration therefore belongs in the upgrade's scope.

Locality describes shared placement domains

The zone-configuration guide at the same commit describes locality as ordered key-value pairs on a node. Possible levels include region, data centre, and rack. CockroachDB attempts to spread replicas according to those levels, with their order determining priority.

For example, two node configurations can provide these locality flag values:

--locality=region=east,datacenter=us-east-1
--locality=region=west,datacenter=us-west-1

The keys and their order must be consistent across nodes. These examples describe two different regions and data centres using the same hierarchy. They are metadata for placement, not a declaration that two nodes alone form a sufficient cluster for any particular availability objective.

When locality is the only placement consideration, the guide recommends setting node locality without adding zone constraints. The allocator then attempts to distribute replicas according to the declared topology. This lets the system use location information while retaining its normal ability to find eligible placement capacity.

Preferences and hard restrictions behave differently

Zone constraints can also refer to node or store attributes, such as storage capability. The guide distinguishes positive preferences from required and prohibited constraints. A positive constraint prefers matching nodes or stores but permits placement elsewhere when matching capacity is unavailable.

A required constraint considers only matching nodes or stores. A prohibited constraint excludes matching nodes or stores. If no eligible capacity remains, either hard restriction can prevent new replicas from being added. That is a different operational outcome from a preference falling back to another location.

This is why a placement rule needs to state whether it describes a desire or an obligation. A preferred storage attribute may reasonably yield to available capacity. A requirement that data remain in a particular country cannot be represented by a preference that silently relaxes when capacity runs out.

I find the explicit distinction useful because placement automation should preserve the meaning of the rule under failure. A hard constraint may leave the system unable to add a replica, but that outcome can be the correct interpretation of a prohibition. Treating every placement request as a preference would hide a different failure of the intended policy.

The binary and build choices are also explicit

The beta notes introduce the CockroachDB Community License and identify make buildoss as the build target for an Apache License 2.0 version. This is a contemporaneous distinction between the distributed product and the open-source build. It should be described using the licensing and build arrangement available now, without projecting another licensing policy onto this release.

Other changes help inspect the running system. Index metadata now includes implicitly added columns, identifying primary-key columns carried in secondary indexes. Nodes also refuse new ranges when they are falling behind on existing work. These changes expose or limit behavior beneath the SQL interface, while the principal deployment changes remain the enforced TLS minimum and a placement model whose preferences and hard constraints have documented consequences when capacity is scarce.