November 10, 2020·6 min read·engineeringcockroachdbstoragebackupsspatial

CockroachDB 20.2 adopts Pebble for local storage

Pebble becomes the default local storage engine while spatial indexing and Core backup support expand the database's uses. Each change has a distinct contract for storage, queries and recovery.

A distributed SQL database still depends on a local storage engine to read and write each replica's data. CockroachDB 20.2 makes Pebble the default for that role. The release catalog dates the production release to November 10. The change concerns the node's persistent key-value implementation, with consequences different from changing the SQL language or the replication protocol.

The release-day summary also introduces spatial support and makes full backups available in Core. These additions address separate needs. Spatial types and indexes let SQL express geographical questions. Backup availability lets more installations use the database's native recovery tooling. Neither should be inferred solely from the storage engine's new default.

I accept CockroachDB's consensus round trips because their cost buys an explicit consistency model. The engine change is interesting at a different level. The local key-value implementation can be designed around the database's actual storage needs while the transaction and replication layers remain responsible for their own guarantees.

Pebble serves the local key-value interface

The 20.2 storage architecture documentation describes Pebble as an embedded key-value store written in Go. It implements a subset of RocksDB's larger feature set and includes optimizations for CockroachDB. The documented intention is compatibility with RocksDB's on-disk format in both directions.

The layer provides atomic write batches and snapshots, which the higher database layers can use. Those facilities are useful building blocks, but they are not the complete distributed SQL transaction model. A local atomic batch and a serializable transaction spanning replicated data have different scopes. The architecture's division of responsibility keeps those scopes identifiable.

Each store has separate engine instances for temporary distributed SQL data and other node data. A block cache is shared across stores on the node. This means a storage-engine decision affects more than the final SQL table files an administrator might imagine. Query execution can use local temporary storage as well as persistent replica storage.

That is why the default is operationally substantive. A node can serve the same SQL workload through a different implementation of local reads, writes and temporary storage. The SQL interface does not provide a complete description of which storage implementation the process selected.

Startup configuration identifies the selected engine

The node-start reference documents storage-engine as a startup choice applying to all stores on a node, including the temporary store. Pebble is the default and RocksDB remains selectable. A deployment can state the choice explicitly with this argument fragment:

--storage-engine=pebble

This is not a complete node startup command. It illustrates the configuration item that distinguishes engine selection from SQL configuration. An existing deployment definition that explicitly selects RocksDB should not be interpreted as selecting Pebble merely because the binary is 20.2. The configured value and the new default are separate evidence.

The on-disk compatibility intention also needs a precise reading. It concerns the formats the engines use. It does not make every configuration or maintenance operation identical between the implementations. A useful deployment record identifies the engine in use alongside the database version, rather than asking a future operator to infer it from a default that may have changed.

I favour that explicitness because an incident involving local storage needs the actual implementation, not only the application's connection string. The engine is an operational dependency even when a SQL developer does not need to select it for each table.

Spatial types define different coordinate models

The spatial data guide distinguishes GEOMETRY from GEOGRAPHY. Geometry represents two-dimensional objects on a plane. Geography represents objects on a spheroidal model, with coordinates interpreted as longitude and latitude. The choice therefore changes the mathematical interpretation of the same-looking pair of numbers.

Points, lines and polygons can describe different application objects within those models. A point representing a warehouse and a polygon representing its service area need operations that respect the selected coordinate system. Storing both as arbitrary strings would preserve their characters without supplying those spatial semantics.

The release includes spatial functions and import paths for established formats. That gives an application more than a place to store coordinates. It can express relationships between spatial objects in SQL and combine them with ordinary relational predicates. The model still needs to match the application's intended units and interpretation of distance.

For someone already using PostgreSQL's substantial extension ecosystem, this is a useful part of the compatibility discussion. Familiar-looking spatial operations are a starting point for examining the documented types and functions. They do not establish that every feature of another spatial system has an identical implementation or availability in this release.

Spatial indexes narrow candidates through coverings

The spatial index documentation describes a decomposition of space using S2 cells. An indexed object receives a covering made of cells, and an inverted index maps cells to objects whose coverings include them. This gives the database a way to identify candidate shapes without inspecting every row.

The covering can include false positives, so a candidate lookup and the exact spatial answer are different steps. A tighter covering can reduce false positives while increasing index entries and the cost of generating or updating them. A larger index can also require more scans. Precision of the covering is consequently a performance tradeoff rather than a free improvement.

That mechanism fits the database's distributed key-value organization. The indexed space has a data-independent subdivision, and objects can map to several cells. Multiple objects can also map to the same cell. The many-to-many mapping explains the inverted representation and why the index's output must still be interpreted as candidates for the spatial predicate.

For an application query, the useful distinction is between asking a geometric question and selecting a candidate lookup strategy. The type and function define the question. The covering and its index support finding the relevant objects without changing the intended exact result.

Core gains native full backups

The full and incremental backup guide makes full backups available to Core and Enterprise users. Incremental backups still require an Enterprise license. A full cluster backup can support restoration at table, database or cluster scope, with the destination and its access configured for the storage being used.

The guide also states a specific restriction on full-cluster restore. The target cluster must never have contained user-created databases or tables. That defines a recovery destination requirement, rather than a general-purpose merge of a saved cluster into any running installation. A procedure needs to identify its restore scope and destination together.

I regard this licensing change as practically useful because a recovery procedure can use native full backups without requiring the Enterprise feature set. It still needs a retained backup and an appropriate target. Replication handles a different failure model from retrieving an independent backup, so the presence of multiple replicas does not make the new capability redundant.

Backup scheduling exposes execution status

The schedule management guide describes creating periodic backup schedules and monitoring their execution. Its metrics distinguish started, succeeded and failed scheduled backups. It also distinguishes waiting for a previous backup from skipping a run because the previous job is still active.

Those states answer different operational questions. A schedule can exist without having produced a successful backup. A job can start without finishing, and a recurrence can arrive while earlier work is still running. The schedule's status and the resulting backup details together explain what recovery material the automation actually produced.

The guide exposes SHOW SCHEDULES for inspecting schedule state and errors, with separate commands for examining the resulting backups. That separation is useful when diagnosing an empty destination or an unexpected gap. It distinguishes failure to trigger work from failure of triggered work and from a mistaken assumption about which backup collection contains the result.

That completes the practical connection between Core backup availability and routine operation. Native support supplies the operation, while scheduling gives it a cadence and visible execution history. The useful recovery record names successful backup artifacts and their intended restoration scope, with enough information to identify which retained data a future restore will consume.