Skip to content

CONCEPT Cited by 1 source

Pull vs push streams

A streaming API is fundamentally either pull-based (the consumer drives — data advances when the consumer asks) or push-based (the producer drives — data advances when the source has more, regardless of consumer state). The axis is load-bearing: it decides how concepts/backpressure works, how cancellation works, how pipelines compose, and how intermediate buffers behave.

The two models

Axis Pull-based Push-based
Who drives Consumer Producer
Evaluation Lazy — runs only when pulled Eager — runs on arrival
Backpressure Implicit (stop pulling = stop producing) Explicit (advisory signal producer consults)
Cancellation Implicit (stop iterating) Explicit (reader.cancel() / abort())
Pipeline buffering Only at pull boundaries Cascades through every stage
Idiom for await…of, iterators, Unix pipes on('data'), enqueue(), RxJS subscribe()

Canonical pull-based: Unix pipes

cat access.log | grep "error" | sort | uniq -c

Data flows left to right. Each stage reads, processes, writes. If uniq -c is slow to consume, sort slows down, grep slows down, cat slows down — all the way back to the disk read. Backpressure is not a mechanism; it's a consequence of the model.

Canonical push-based: Web streams TransformStream

const t = new TransformStream({
  transform(chunk, controller) {
    controller.enqueue(processChunk(chunk));
  }
});

source.pipeThrough(t).pipeTo(destination);

transform() runs on write, not on read. The moment a chunk arrives upstream, it's processed and enqueued downstream, regardless of whether the destination is ready. If the destination is slow, chunks accumulate in the readable side's buffer. If the transform is fast and synchronous, no backpressure is signaled back to the writable side at all.

In a 3-transform chain, six internal buffers can be filling simultaneously before the final consumer begins pulling.

Why Web streams are push-biased

The WHATWG Streams Standard was designed 2014-2016, before async iteration landed ES2018. Without for await…of, the natural model for "a sequence of things arriving over time" was producer-callback: ReadableStreamDefaultController.enqueue(). Async iteration was retrofitted — ReadableStream[@@asyncIterator] now exists — but the underlying machinery (readers, locks, controllers, pipeThrough) is still push-oriented. "Data cascades through [intermediate] buffers in a push-oriented fashion."

Why pull-based matters for performance

Each layer of push-based evaluation creates work that cannot be skipped:

  • Allocation — one { value, done } result object per read().
  • Promise machinery — one internal promise per enqueue/read pair; plus coordination promises for backpressure signaling.
  • Intermediate buffers — each stage holds its own queue; bytes sit in N queues in series.

Cloudflare's 2026-02-27 benchmark of a 3× transform chain measured ~80-90× faster for a pull-based design vs Web streams "because pull-through semantics eliminate the intermediate buffering that plagues Web streams pipelines".

Why pull-based matters for correctness

Push-based: if the consumer disconnects and the producer isn't watching, the producer keeps running — leaking CPU, memory, network. fetch() response bodies that aren't consumed or cancelled have caused connection pool exhaustion in Node's undici, because the stream holds a reference to the underlying connection until GC runs.

Pull-based: if the consumer stops iterating, next() stops being called; the producer's yield blocks; nothing runs. The resource is held only as long as someone is reading.

Mixed / adapter surfaces

Real systems often have a pull consumer on one side and a push producer on the other — ReadableStream (push) → async iterator (pull). The adapter layer is the cost surface: it must either (a) buffer eagerly (losing pull semantics), or (b) convert push callbacks into pull primitives (introducing scheduling overhead per chunk).

systems/new-streams' thesis is that a streaming API should be pull-based end-to-end by default, with adapters to push producers when bridging legacy APIs (ReadableStream.from(…) in the other direction).

Seen in

Last updated · 200 distilled / 1,178 read