Skip to content

CONCEPT Cited by 1 source

Reactive streams

Reactive Streams is an initiative and specification providing a standard for asynchronous stream processing with non-blocking backpressure on the JVM. The core interfaces (Publisher, Subscriber, Subscription, Processor) define a protocol where downstream consumers pull demand from upstream producers — the inversion of control from traditional push-based streaming.

Why it matters

At scale (millions of records/sec), the three naïve responses to fast-producer/slow-consumer are all unacceptable:

  1. Unbounded queues — buffer until OOM, then crash.
  2. Drop-based flow control — discard data, lose completeness.
  3. Batch processing — process later, lose freshness.

Reactive Streams provides a fourth option: slow down gracefully under load without losing data. When a downstream stage can't keep up, it signals upstream to reduce emission rate. The signal propagates through the entire pipeline.

Complexity trade-offs

Netflix's Service Topology team (2026-07-13 post) documents the learning costs:

  • Non-intuitive demand model — traditional code flows top-to-bottom; reactive streams are demand-driven from the consumer side.
  • Async boundary complexity — the .async operator creates thread boundaries that improve parallelism but introduce buffer sizing, demand signalling, and error propagation complexity.
  • Silent stalls — when a stream stops processing, there is no stack trace. Diagnosis requires understanding internal demand signals, buffer states, and materializer state.
  • Over-parallelization risk — too many .async boundaries create more overhead than benefit.

(Source: sources/2026-07-13-netflix-building-service-topology-at-scale-architecture-challenges)

Implementations

  • systems/apache-pekko (Pekko Streams) — used by Netflix Service Topology
  • Akka Streams (pre-fork, BSL-licensed)
  • Project Reactor (Spring WebFlux)
  • RxJava (with Flowable)

Seen in

Last updated · 585 distilled / 1,765 read