Skip to content

SYSTEM Cited by 1 source

Redpanda Shadowing

Shadowing is a Redpanda 25.3 feature (2025-11-06 preview) that creates a byte-for-byte, offset- preserving, hot-standby clone of a source Redpanda cluster in a different region. Shadowing is positioned as Redpanda's first-party answer to cross-region disaster recovery for the streaming-broker substrate, replacing the connector-mediated shape of MirrorMaker2 and the prior Redpanda Migrator.

Canonical verbatim from the sources/2025-11-06-redpanda-253-delivers-near-instant-disaster-recovery-and-more|25.3 launch post:

"Shadowing creates a fully functional, hot-standby clone of your entire Redpanda cluster — topics, configs, consumer group offsets, ACLs, schemas — the works!"

"Shadowing is built into the Redpanda broker itself and uses the standard Kafka API to link clusters. No MirrorMaker 2 or Redpanda Migrator connectors are used under the hood."

Load-bearing properties

  1. Byte-for-byte, offset-preserving replication. The shadow cluster holds the same per-partition offsets as the source. Consumers that fail over to the shadow resume at the same offsets they held on the source, with no offset-translation bookkeeping. Contrast MirrorMaker2 which maintains a per-consumer-group offset-translation map because the two clusters produce messages with independent offset numbering.

  2. Broker-internal. The replication mechanism lives inside the Redpanda broker and uses the Kafka wire protocol to link clusters, not a separate Kafka Connect cluster with MM2 connectors. See concepts/broker-internal-cross-cluster-replication.

  3. Asynchronous. "Shadowing combines an asynchronous replication mechanism with offset preservation." Writes on the source are not blocked on shadow-cluster acknowledgement; the shadow catches up on its own schedule.

  4. Full-cluster clone, not a topic-level mirror. The clone covers "topics, configs, consumer group offsets, ACLs, schemas — the works". This is stronger than MM2's topic-granularity mirroring + separate schema-registry replication + separate ACL replication — all in one feature.

  5. Seconds-range RPO/RTO. "Recovery Point Objectives (RPOs) measured in a few seconds and similar Recovery Time Objectives (RTOs), limited only by timeout settings for producers and consumers." Positioned between stretch cluster RPO=0 and MM2 non-zero-RPO-dependent-on-lag.

Setup surface

"Any cluster can create a shadow link to a source cluster. With a few lines of configuration or using the interactive process in Redpanda Console, you can enable a shadow link in seconds to start replicating your most critical workflows."

The Redpanda Console exposes a monitoring view for lag + throughput on the shadow cluster (verbatim "Monitoring a shadow cluster in Redpanda Console" from the post).

Position in the Redpanda DR axis

Redpanda's cross-region DR / HA axis now has three shapes:

Shape RPO RTO Cost Complexity
Stretch cluster 0 Very low (Raft re-election) Per-write cross-region RTT + egress Single cluster
Shadowing Seconds Seconds (client timeout-bound) Async bandwidth + 2× cluster cost Two clusters, single feature
MirrorMaker2 Non-zero (lag) Connector restart + offset-translation 2× cluster + MM2 infra Highest (Kafka Connect cluster)

Shadowing occupies the "two-cluster DR without the connector overhead" slot — offset preservation removes the client-side failover friction that's MM2's load-bearing operational cost; broker-internal implementation removes the Kafka-Connect operational surface.

Contrast with shadow-cluster (release-validation sense)

Not to be confused with shadow cluster in the release-validation sense (Meta Presto's shadow cluster for catching post-compilation regressions on long-running queries). Redpanda's Shadowing is a DR mechanism; the Presto-style shadow cluster is a pre-promotion validation mechanism. Same word, different pattern family.

Target workload

"Designed for workloads with high-throughput, low-latency requirements and can operate with a low, but nonzero, RPO and RTO." — i.e. the same write profile stretch cluster targets but with independent per-region clusters instead of one stretched cluster.

Status (2025-11-06)

Preview / coming-soon at publication — the 25.3 release is "coming soon" per the post. Beta/GA classification of Shadowing specifically is not stated.

Seen in

Last updated · 470 distilled / 1,213 read