Skip to content

PATTERN Cited by 1 source

Dual-write migration

Dual-write migration is a pattern for protocol/backend transitions where producers emit to both the legacy and the new system simultaneously for the overlap window. The new system is validated against the old one in production with real traffic; cutover and decommission happen only after it's proven out.

Why it works for protocol migrations

  • No big-bang switch. The old pipeline stays whole-stack authoritative until dashboards/alerts are validated against the new one.
  • Low per-team friction when the dual-write is done in a shared platform library: one library change, many services migrated.
  • Immediate visibility into scale issues. Emission volume is full production from day one, so bottlenecks surface before you're depending on the new pipeline.

Airbnb made a shared metrics library dual-emit StatsD + OTLP to migrate ~40% of services (those on the shared lib) with a single config change; only the stragglers needed per-service work. (Source: sources/2026-04-16-airbnb-statsd-to-otel-metrics-pipeline)

Costs

  • 2× emission load during overlap. Can itself become the thing that breaks — Airbnb's highest-cardinality emitters regressed under the combined OTLP+StatsD load and had to move to delta temporality (concepts/metric-temporality).
  • 2× storage / ingest cost downstream for the migration window.
  • Consistency work if the two systems disagree (different aggregation semantics, different retention) — differences need to be explained, not hidden.

Where to implement the fork

Best place: a shared instrumentation library used by many services. One change, wide coverage, easy rollback. Second-best: a collector sidecar that can translate between protocols (but a translation layer adds CPU and a format-mismatch failure mode — Airbnb specifically called out removing StatsD→OTLP translation from the OTel Collector as a win of native OTLP emission).

Exit criteria (before removing the legacy path)

  • All dashboards and alerts ported and validated against the new system.
  • Query-result equivalence checked on a sample of high-priority metrics.
  • Consumer tooling (LLMs, runbooks) updated to the new data model.
  • A documented rollback plan, in case the new pipeline surprises you after cutover.

Seen in

Last updated · 200 distilled / 1,178 read