April 4, 2018·6 min read·engineeringcockroachdbpartitioningjsonb

CockroachDB 2.0 makes data placement an application decision

CockroachDB 2.0 adds partition placement controls and JSONB indexing. The release exposes decisions about geography and document structure while preserving a SQL interface.

Distributing a database does not determine where each customer's records should live. An even distribution can balance storage while leaving a customer far from the replicas needed for an operation. CockroachDB 2.0 gives applications more control over that distinction through table partitioning. Its other prominent addition, JSONB, similarly gives applications more choice about how a record is represented inside a relational database.

The April 4 release announcement presents geo-partitioning as control over data placement at the record level. It also introduces PostgreSQL-compatible JSON operations and inverted indexes. These capabilities solve different problems, but both make application knowledge relevant to database organization. Geography and document structure are properties the application often knows before the database can infer their importance.

I appreciate CockroachDB's default serializable isolation and accept that consensus adds latency. Placement control matters within that tradeoff. A consistent distributed database still needs replicas in locations that suit the application's access pattern and failure requirements. The new control makes those choices more deliberate without changing the need to reason about coordination.

Partitioning supplies a placement boundary

The 2.0 release notes classify table partitioning as an enterprise feature. They describe using it to keep records near users or to place infrequently accessed records on different storage. Partitioning supplies a subdivision of the table to which placement policy can apply. A row's values become relevant to where its data is stored.

That relationship has a schema consequence. The application needs a reliable attribute describing the grouping it wants to place. If geography belongs to a customer, the schema needs a way to associate the relevant records with that geography. A placement rule cannot compensate for missing or ambiguous application data about which group a record belongs to.

An application's notion of locality also needs to cover the records its operations use together. Keeping one customer table close to a user may leave another frequently joined table elsewhere. The transaction then still crosses the geographical boundary. Placement decisions become more useful when they follow complete operations instead of isolated table sizes.

This is an engineering implication of record-level placement, not a promise that partitioning eliminates all remote work. Some operations intentionally combine customers or regions. A global report may need information from many placements even when individual customer requests are local. The benefit depends on which operations the application's placement policy favors.

The March geo-partitioning explanation illustrates two replicas in Asia and another elsewhere. After losing one Asian replica, reads can remain local while writes need the remaining remote replica for consensus. The example makes a useful distinction between the normal placement and the communication required after a failure.

A failure-tolerant layout can therefore have different latency characteristics in different states. Retaining availability may require communication that normal operation avoids. That is not a contradiction in the placement policy. It is a consequence of which replicas remain and which acknowledgements the operation requires.

Locality does not remove a failure-domain decision

The same example emphasizes diversity among replica locations. A placement chosen solely for short communication can put too much responsibility in one failure domain. A placement chosen solely for geographic separation can impose remote communication on common operations. The operator needs a policy that expresses the desired relationship between those properties.

I find explicit placement useful because it makes that relationship discussable. A configuration can be read against the application's requirements. A vague claim that a database is multi-region does not identify which region's failure it should survive or which operations remain local while it does so. Those are separate questions about a specific deployment.

The release also adds a geographical node map as an enterprise capability. A map does not establish that a placement policy is correct, but it gives the operator another view of the topology being managed. The policy and the observed deployment need to agree for the intended failure behavior to be plausible.

JSONB adds queryable variation within a row

Justin Jaffray and Andy Woods's March JSONB article explains the choice of an encoded binary representation with PostgreSQL-style operations. Values can be accessed without decoding the entire document. The article also describes inverted indexing by paths from a document's root to its leaves, enabling containment queries without predeclaring every field as a separate relational column.

For a column named metadata, containment can express a partial document match. This simple expression uses the operation described in that article:

SELECT '{"kind":"sensor","enabled":true}'::JSONB
       @> '{"kind":"sensor"}'::JSONB;

The expression asks whether the larger document contains the specified structure. It does not require string matching against the serialized document. That distinction lets the database reason about a JSON value through its operators instead of treating it as opaque application text.

The data-model choice remains consequential. A field stored in a JSON document has a different role from a declared column with an explicit relational constraint. Flexibility can be useful for attributes that vary among records. It should not be confused with an absence of structure. The application still assigns meaning to those fields and depends on that meaning when it queries them.

An inverted index changes which questions can use an access path, but it also creates derived data to maintain. A document with many indexed paths can contribute multiple entries. The useful question is which containment predicates the application needs, together with the update pattern of the documents carrying them. Supporting changing document shapes does not make indexing independent of the data being written.

The JSONB article also discusses write amplification under MVCC and replication. Updating a large document repeatedly can require retaining previous versions and replicating changed data. That gives the choice between a JSON attribute and a relational column an operational dimension. Data flexibility and update granularity affect different parts of the workload.

Computed columns, also listed among the release's core additions, store values generated by expressions over other columns. They offer another way to represent a derived attribute that an application repeatedly needs. This is relevant when a document contains a value that has acquired a more stable role in the application's query patterns. The data model can expose that role explicitly without requiring every optional attribute to become an independent column at once.

The consequence is a broader set of modeling choices within one database. An application can retain a varying document while giving selected information a deliberate relational representation. That representation needs a maintained expression with the intended meaning. Moving extraction into a schema definition makes the dependency visible there, and any later change to the document's conventions needs to account for it. The database can maintain the expression, but the application still defines which document shapes it considers valid.

The release also changes operational interfaces

The release notes introduce /health?ready=1, which returns an unavailable response when a node is shutting down or being decommissioned and cannot accept SQL work. A live process is not necessarily a suitable target for a new connection. This endpoint gives load-balancing configuration a way to express that distinction during operations such as rolling upgrades.

That is another case where a broad capability needs a more precise interface. A process check answers whether the process responds. A readiness check answers whether it should receive the work being routed. Confusing the two can send new requests toward a node deliberately leaving service, even while the operator is following the intended maintenance procedure.

Role-based access control is also listed as enterprise functionality. It groups SQL privileges through roles instead of requiring separate grants for every user. As with placement, the grouping is valuable when it corresponds to an application or operational responsibility. A role name alone does not establish that its included privileges match that responsibility.

The release therefore expands more than the query language. It gives operators and applications new ways to describe where records belong, how semi-structured values are queried, and whether a node should receive work. The edition boundaries matter to anyone planning to use those capabilities, since JSONB is a core feature while partitioning and roles require an enterprise license in this release.

My interest in 2.0 is the ability to keep those decisions inside a database that continues to expose SQL and strong transaction semantics. The application still needs a coherent locality model and an intentional document model. The new capabilities let those models guide concrete storage and execution behavior, including the remote coordination a transaction needs after a replica becomes unavailable.