Skip to content

PATTERN Cited by 1 source

API as single source of truth over event streams

Problem

At sufficient microservices scale, the common "subscribe to the event stream and build your own local store" architecture produces competing sources of truth:

  • Teams with capacity consume events and build custom stores, each with its own transformation.
  • Teams without capacity depend on whichever upstream team's store they can read from, inheriting 2–3 hops of data massaging.
  • New consumers have no canonical answer to "where do I get this data?"
  • Data lineage is lost; downstream representations drift silently from the producer's intent.

In Zalando's framing:

"A simple request—'I'm building a new feature and need access to product data. Where do I get it?'—had an unreasonable answer: 'Subscribe to our event stream, replay events from the dawn of time, and build your own local store.'" (Source: sources/2025-03-06-zalando-from-event-driven-chaos-to-a-blazingly-fast-serving-api.)

Pattern

Build a canonical read API that owns all legacy representations of the entity and is fast enough that consumers stop maintaining local copies. Keep the event-driven substrate on the producer side, but route consumer access through the API. The API becomes the single source of truth; event streams become an implementation detail of ingestion.

Key elements:

  • Ownership of all representations. The API accepts Accept headers (or equivalent) for every legacy format so consumers don't have to re-derive the shapes they were already depending on. (See patterns/accept-header-format-negotiation-for-legacy-sunset.)
  • Legacy-format emission back onto event streams. For sunset window, the new API re-emits legacy formats onto the old event streams so legacy producers can be decommissioned immediately while consumers migrate on their own schedule. (See concepts/legacy-format-emission-for-incremental-sunset.)
  • Performance that justifies migration. The API has to match or beat every team's local cache to make migrating obviously-correct. For Zalando's PRAPI: single-GET P99 sub-10 ms on ~1,000-line JSON payloads — comparable to or better than local in-memory caches at the consumer team's altitude.
  • Central ownership of the hot-data model. A small team (Product Data Serving, in Zalando's case) owns the API, the canonical shape, and the transforms. Other teams stop maintaining their own.

Why not just "use events"

Event-driven is still the producer-side default. What this pattern rejects is event-driven as the consumer access model. The ergonomics of per-consumer event consumption break down at 350+ teams:

  • Every consumer maintains partition assignment, offset tracking, replay logic.
  • Bootstrapping a new consumer means replaying the full history.
  • Every consumer independently implements the same transformations from the shared schema.
  • Schema evolution requires coordinating N consumers.

A canonical read API collapses N local-stores into 1 managed store. Consumers that still need bespoke projections can project from the API, not from the events.

When to reach for this

  • Dozens+ of internal consumers of the same domain entity, each with its own derived store.
  • Drift complaints — downstream consumers disagree about field meaning, values are mismatched.
  • Peak-event pipeline bottleneck — your event pipeline is the single thing slowing down during high-traffic periods because too many stages are recomputing the same composed entity.
  • Slow onboarding — new teams take weeks to bootstrap a consumer of a shared entity.

When event-driven consumer access is fine

  • Low consumer count. A handful of known downstream systems can coordinate schemas directly.
  • Strong need for at-most-once or deterministic replay semantics. Event streams give you these for free; an API may or may not.
  • Derived workflows that fundamentally need events (stream-processing, audit, real-time analytics). Keep the events for these; add the API for the serving path.

Seen in

Last updated · 501 distilled / 1,218 read