CockroachDB changefeeds can choose Kafka partition hashing

Two producers can agree on a message key and still send it to different Kafka partitions. CockroachDB 26.1.1, released on March 9, adds a partition_alg changefeed option that makes the hashing choice explicit. The release notes list fnv-1a as the default and murmur2 as the alternative. The option applies only to Kafka v2 sinks and is controlled by changefeed.partition_alg.enabled.

This is a compatibility decision about where records go. Apache Kafka's Java producer implementation uses Murmur2 for its default keyed partition calculation. CockroachDB's new option therefore makes it possible to select the same named algorithm instead of being restricted to its existing FNV-1a default. That removes one source of disagreement between producers, though it does not establish that their complete partition calculations are identical.

I value CockroachDB's explicit consistency guarantees, even when they carry a visible latency cost. The same preference applies to a changefeed's downstream behavior. A partition decision is easier to reason about when the configuration states which calculation the producer uses. Agreement inside the database does not automatically establish agreement about where independently produced Kafka records will be consumed.

The serialized key is part of the calculation

Kafka's Java code makes the inputs concrete. Its partitionForKey method accepts serialized key bytes and a partition count. It computes Murmur2 over those bytes, converts the result to a positive value and takes the remainder modulo the number of partitions. The logical value that an application calls a key is therefore only an earlier input to the process.

Two representations of the same business identifier can have different bytes. A producer that serializes an identifier as text and another that includes it in a structured representation have not necessarily supplied the hash function with the same input. Selecting Murmur2 on both sides addresses the algorithm choice, while comparing actual serialized keys addresses a separate compatibility condition.

The partition count is another explicit input in the Java implementation. Matching an algorithm does not make a partition number independent of that count. This matters when evaluating whether two streams can be processed by corresponding consumers. Their routing contract includes the topic arrangement and the key representation, as well as the name of the hash.

That is why the CockroachDB setting should be understood as one specific control. The release notes establish the available algorithms and sink restriction. They do not promise automatic alignment with every Kafka producer or serializer. A compatibility claim needs to identify the producer behavior being matched, rather than treating the word Kafka as a complete partitioning specification.

The feed retains its selected algorithm

The release notes give the following form for selecting Murmur2. This example assumes an existing events table and a configured Kafka v2 sink whose connection URI replaces the illustrative endpoint.

CREATE CHANGEFEED FOR events
INTO 'kafka://broker.example:9092'
WITH partition_alg = 'murmur2';

The cluster setting controls access to the option, but disabling it later does not reset the algorithm of an existing feed. A feed created with a particular choice continues using it unless altered to select another value. This is an important persistence rule for an operator who treats the cluster setting as an emergency feature switch.

Turning off that switch prevents neither the existence of the stored choice nor its continued use as described by the notes. The changefeed definition remains the relevant configuration for understanding its routing. An operational record needs the algorithm attached to each feed, rather than only the current value of the cluster-level enablement setting.

Changing a feed's algorithm can change partition placement because the calculation changes. That makes it a downstream compatibility change, even if the emitted row contents remain the same. Consumers that rely on routing need a transition plan appropriate to their own processing model. The release does not define a universal migration procedure for combining records produced under different hash choices.

Key selection is a separate option

CockroachDB's changefeed documentation also describes key_column, which overrides the key used to determine the downstream partition. That setting changes what gets hashed. The new partition_alg setting changes the selected hashing algorithm. Combining the two without distinguishing their roles would make it difficult to explain why a record moved.

The documentation says key_column requires the unordered option and does not preserve message ordering from CockroachDB to the sink. Choosing a hash does not remove that qualification. An application must keep the changefeed's ordering rules separate from its desire to place related records together. The correct partition number alone does not prove every ordering property a consumer might expect.

Version 26.1.1 has a distribution qualification too. Its notes say availability is limited to selected CockroachDB Cloud clusters, with self-hosted upgrade requests directed to support. For an integration depending on the new option, that means the March announcement describes both a new routing control and a release whose availability must be established for the target deployment.