PATTERN Cited by 1 source
FIFO buffer overload protection¶
Definition¶
A two-layer overload protection pattern for high fan-out services: an outer overload filter sheds brand-new inbound requests (503 + Retry-After) when outbound in-flight crosses a hard cap, while an inner FIFO buffer queues already-admitted fan-out calls just beneath that cap. The queue self-drains as in-flight completes — no timer, no polling.
How It Works¶
- A batch request arrives and is admitted (hasn't hit the overload cap).
- The batch unpacks into N parallel downstream calls.
- If outbound in-flight is at the hard cap, those calls enter the FIFO buffer instead of hitting the wire.
- As in-flight requests complete, the next queued fan-out is released.
- If a new batch request arrives while the cap is already hit, the overload filter rejects it immediately with 503.
This prevents already-admitted work from being wasted while stopping new work from compounding a spike.
Seen In¶
- systems/zalando-prapi — protects the products fan-out path; prevents node-freeze events from cascading into retry storms (Source: sources/2026-06-22-zalando-client-side-load-balancing)