PATTERN Cited by 1 source

Queue isolation per pipeline

Pattern

Give each logical pipeline (event surface, workload class) its own physically separate queue and independently deployed worker fleet. A burst in one pipeline can only consume its own fleet's capacity; it has no path to slow down another pipeline's pods because they are different deployments entirely.

Problem it solves

A single delivery queue means one workload's traffic waits behind another's — classic concepts/head-of-line-blocking. Even with priority queues or weighted fair queueing, a truly bursty workload on a shared queue can create unpredictable delays for unrelated traffic.

Design considerations

• What separates pipelines isn't a "fast vs slow" label — it's the operational characteristics each surface needs: different timeout budgets, retry policies, and failure-recovery shapes.
• Granularity trade-off: too coarse → unrelated workloads still share a fleet and interfere; too fine → operational overhead of many fleets that rarely see contention.
• One surface's worst day stays inside its own boundary — the core isolation guarantee.

Seen in

• sources/2026-07-29-atlassian-inside-the-events-rail — separate queues + worker fleets for product events, lifecycle events, Bitbucket repo hooks, audit-log delivery, workflow events. "Mixing surfaces with different semantics into a single queue is one of the cleanest ways to create silent failures."