Skip to content

CONCEPT Cited by 2 sources

MirrorMaker2 async replication

Definition

MirrorMaker 2 (MM2) is Apache Kafka's canonical cross-cluster asynchronous replication tool — a Kafka Connect- based application that consumes from a source cluster and produces to a destination cluster, keeping topic data continuously mirrored between two independent Kafka (or Kafka-API- compatible) clusters. The replication is asynchronous: a produce to the source cluster is acknowledged locally, and MM2 copies the record to the destination cluster on its own schedule.

MM2 is the canonical alternative shape to a multi-region stretch cluster for cross-region HA/DR: two clusters + async mirror rather than one cluster across regions with sync quorum.

Canonical Redpanda framing (for comparison against stretch cluster)

"Unlike in asynchronous replication, where you have two separate clusters with MirrorMaker2 replication between them and a non-zero RPO, multi-region clusters have RPO=0 and very low RTO when there is a region-level outage. This is because new leaders are automatically elected — as part of the Raft protocol — in the surviving regions when any region goes down."

"If strong consistency is not an absolute requirement but availability is, at the expense of slightly older data, multiple independent Redpanda clusters across different regions with MM2 replication can be set up. This prioritizes cluster availability allowing regions to operate independently but can lead to slightly older data that is a factor of how quickly replication can occur."

(Source: sources/2025-02-11-redpanda-high-availability-deployment-multi-region-stretch-clusters)

Trade-offs vs stretch cluster

Dimension MM2 async (two clusters) Stretch cluster (concepts/multi-region-stretch-cluster)
RPO Non-zero (= replication lag at failure) 0
Write latency Local-region (no cross-region wait) Cross-region quorum RTT
Write availability during region loss Each cluster stays writable Minority-region writes fail (Raft quorum lost)
Consistency between clusters Eventual Strong (single Raft group spans regions)
Control plane Two independent Single
Operational complexity Higher (two clusters + MM2 infra) Lower
Compute cost 2× base cluster + MM2 connectors 1× cluster with cross-region replicas
Cross-region bandwidth One replication stream per mirrored topic Full replication factor × write volume per region pair

MM2 optimises for per-cluster availability and write latency at the expense of cross-cluster consistency and RPO. The stretch cluster optimises for RPO = 0 and single-cluster simplicity at the expense of per-write cross-region latency.

How MM2 actually works (short mechanism)

  • Connector-based: MM2 runs as Kafka Connect source + destination connectors. The source connector consumes from topics in the source cluster; the destination connector produces to renamed topics in the destination cluster (e.g., source-cluster- name.topic-name is the default remote-topic naming scheme).
  • Offset translation: MM2 maintains per-consumer-group offset translations between the two clusters so consumers can fail over from source-cluster offsets to destination-cluster offsets.
  • Bidirectional: MM2 can run in both directions between two clusters (A → B and B → A), forming active-active topologies with careful naming conventions to avoid infinite replication loops.

Where MM2 applies vs stretch cluster

Pick MM2 when:

  • Write SLA is tight (sub-cross-region-RTT); the application cannot tolerate cross-region quorum per write.
  • Availability within a region is more important than cross-region consistency (e.g., regional business units operating mostly independently, with some shared topics for cross-region analytics).
  • Cross-region bandwidth must be minimised to one replication stream per mirrored topic rather than full replication factor.
  • Operational model already has per-region deployment pipelines.

Pick stretch cluster when:

  • RPO = 0 is a hard requirement for the workload (financial ledger, mission-critical event log where losing any ack'd write is unacceptable).
  • Single-control-plane operations outweigh per-write latency cost.
  • Latency budget allows cross-region quorum (regional stretch, not transoceanic).
  • Client regions can be accommodated with leader pinning + follower fetching so the client-facing hops stay intra-region and only the replication- side hops are cross-region.

Distinction from remote read replica

concepts/remote-read-replica-topic is not a MM2 replacement — remote read replica is a read-only mirror backed by object storage. MM2 mirrors both reads and writes (in an active-active setup) and is a full two-cluster replication mechanism.

Displacement by broker-native Shadowing (Redpanda 25.3)

Redpanda Shadowing (2025-11-06 preview) is Redpanda's first-party broker-native replacement for MM2-style cross-cluster replication, explicit in the launch post:

"Shadowing is built into the Redpanda broker itself and uses the standard Kafka API to link clusters. No MirrorMaker 2 or Redpanda Migrator connectors are used under the hood."

(Source: sources/2025-11-06-redpanda-253-delivers-near-instant-disaster-recovery-and-more)

Shadowing displaces MM2 on three axes simultaneously:

  1. Broker-internal — no separate Kafka Connect cluster to operate.
  2. Offset-preserving — consumers fail over at the same offsets they held on the source; no offset-translation map, no consumer re-snapshot.
  3. Seconds-scale RPO/RTO — async-replicated hot-standby clone (full cluster state: topics, configs, ACLs, schemas, consumer-group offsets), with RTO "limited only by timeout settings for producers and consumers".

MM2's remaining strength is Kafka-API-ecosystem neutrality — it works between any two Kafka-API-compatible brokers regardless of vendor (Kafka ↔ Kafka, Kafka ↔ Redpanda, Kafka ↔ WarpStream, etc.). Shadowing is Redpanda-to-Redpanda only. See broker-internal cross-cluster replication for the full trade-off analysis.

Seen in

Last updated · 470 distilled / 1,213 read