MirrorMaker 2 gives cross-cluster replication an offset translation model

Copying a Kafka record to another cluster does not preserve its offset as a universal address. The destination appends the record to a different log, and a consumer's source position needs a meaningful destination equivalent. Kafka 2.4, released December 16, introduces MirrorMaker 2 with an explicit cross-cluster model for records and consumer progress. That is a more useful change than describing it only as a new replication command.

The README shipped with 2.4.0 identifies a framework built on Kafka Connect. It replicates topics and consumer-group information, preserves partitioning, and synchronizes cross-cluster offsets. Remote topic naming distinguishes the origin of replicated data. The new framework supplies metadata about the copy operation as well as the copied records.

I value Kafka because a reader's position gives it an explicit relationship to durable history. Moving that reader between clusters requires preserving the meaning of its progress. A bare numeric offset is insufficient when the records now occupy another log.

Replication between clusters differs from partition replicas

Kafka's 2.4 operations documentation calls copying between clusters mirroring to distinguish it from replication among brokers inside one cluster. It describes the older MirrorMaker as a consumer and producer connected together. The source and destination clusters are independent, and their offsets need not match.

That independence is useful for deployment. Separate clusters can operate in different datacenters while a mirroring process carries records between them. A failure of the inter-datacenter connection makes the mirror fall behind until communication returns. This is a different arrangement from making every broker participate in a single cluster across the same link.

MirrorMaker 2 retains a directional model. A flow from cluster A to cluster B is a separately configured operation. Fan-out, aggregation and active-active arrangements combine several such flows. Calling an arrangement active-active therefore describes the enabled directions, not a single global partition order spanning independent clusters.

The README requires each flow to be enabled explicitly. Listing two cluster endpoints does not start bidirectional replication automatically. This makes the intended data movement inspectable in configuration, including which topics and consumer groups belong to each direction.

Remote topic names retain source identity

By default, a source topic becomes a remote topic with the source cluster alias prefixed to its name. A topic named orders from cluster west becomes west.orders at the destination. The naming scheme prevents records originating from different clusters from being written into the same destination partition merely because their local topics share a name.

This is particularly relevant in an aggregation topology. Several clusters can each have a locally meaningful orders topic. Their records remain attributable to the appropriate source when copied to the aggregation cluster. The shared suffix does not imply shared offset numbering or one combined source history.

The README permits customizing the separator or supplying a ReplicationPolicy implementation. That is a policy extension for how remote topics are defined. The default behavior remains important to an application moving its reads because the destination topic name can differ from the source topic name.

Partitioning is preserved by the new framework, according to the tagged README. That gives a corresponding partition relationship for records copied from a source topic. It still does not make the destination's offset numbers equal to the source's numbers. Retaining the partition mapping and translating positions are separate requirements.

A flow configuration identifies both endpoints

The supplied 2.4.0 sample configuration names cluster aliases and their bootstrap servers, then enables individual flows. A reduced example with placeholder hosts illustrates a single direction.

clusters = west, east
west.bootstrap.servers = west-broker.example:9092
east.bootstrap.servers = east-broker.example:9092
west->east.enabled = true
west->east.topics = orders

The aliases are names used by MirrorMaker's configuration and replication policy. They do not replace the broker endpoints supplied on the corresponding properties. The explicit direction says that this example copies the selected topic from west to east, without independently enabling an east-to-west flow.

The documented launch command starts the Connect-based mirroring process with that properties file.

bin/connect-mirror-maker.sh mm2.properties

Multiple MirrorMaker processes handling the same clusters discover each other and share work. The README also warns that their configuration must agree for flows into the same target cluster. Two processes independently configured for different topic selections on the same flow do not automatically combine those selections. The elected leader's configuration can determine which selection applies.

That detail changes the operating model from a collection of unrelated copy commands. The processes cooperate through Connect and the target cluster, so configuration is shared state. A deployment needs one consistent definition of the flow across the participating processes.

Checkpoints translate consumer progress

The 2.4.0 checkpoint task reads source consumer-group offsets through the administrative client. For relevant topic partitions it translates the upstream offset into a downstream offset using the offset synchronization store. It then creates a checkpoint containing the group, renamed topic partition, both offset values and offset metadata.

That is more precise than copying the source cluster's consumer-offset topic byte for byte. The source value is meaningful within its source partition. The checkpoint records a translated position for the corresponding destination partition. Topic renaming is included in that translation, so the destination's name is part of the usable result.

The task skips translated downstream offsets that fail its validity condition. Its source comment identifies these as offsets it cannot translate accurately. This makes the translation dependent on available synchronization information. The framework does not invent a matching destination position merely because it can read a source group's committed number.

Checkpoint emission also has an interval. The README exposes emit.checkpoints.interval.seconds as a setting. A checkpoint is therefore an observation carried through the replication system, not a synchronous update performed at the same instant as every source consumer commit. The age and availability of that information matter when using it to establish destination progress.

Reading a translated offset does not move the application

The MirrorClient implementation at 2.4.0 exposes remoteConsumerOffsets. Its documentation describes translating a remote group's positions to corresponding local offsets, automatically renaming topics according to the replication policy. The implementation reads the checkpoint topic and returns matching group entries as a map.

The method returns information. It does not, in this implementation, redirect the application's network connection or start the consumer on the other cluster. A complete move still needs an application-side decision about when to stop using the source and how to apply the translated destination position.

This distinction also limits the phrase consumer-group replication. The useful delivery is the metadata needed to understand a group's equivalent position elsewhere. It should not be described as automatic transparent failover of every application that uses the group. The tagged API shows the separate consumption of checkpoints and return of translated positions.

My preference is to preserve that distinction when reasoning about recovery. Replicating history and identifying a restart position are necessary pieces. The application's side effects and the timing of its move remain part of its own processing contract. Returning a position map cannot make those operations atomic across independent systems.

Data movement has its own placement and observations

The README recommends keeping MirrorMaker producers near their target clusters. Its --clusters option identifies nearby clusters for a process and prevents that process from producing records or configuration to distant targets. The reason given is avoiding producer lag and unnecessary transfer costs. The direction of the network-heavy work consequently matters even though either endpoint contains Kafka brokers.

The framework inherits Connect metrics and adds metrics for replicated record age and propagation latency. It also measures checkpoint latency. Those observations address different stages, the movement of records and the movement of consumer-progress information. A current destination topic alone does not prove that a usable checkpoint has caught up to the latest source group position.

MirrorMaker 2 gives those relationships explicit names and APIs. A flow identifies the source and destination, a remote topic retains source identity, and a checkpoint relates consumer positions across their different logs. The destination remains an independent Kafka cluster whose offsets have their own meaning, with translation supplying the connection a moving reader needs.