Skip to content

PATTERN Cited by 1 source

Durable job queue for agent reliability

Pattern

When an AI agent system must stay responsive through infrastructure disruptions (deploys, restarts, worker crashes), back the job dispatch layer with a durable queue (typically Postgres-backed) rather than an in-memory queue. Diagnostic/reasoning passes become first-class persisted jobs that survive any single-worker failure.

Key properties: - Crash-safe: if a worker fails mid-pass, another picks the work up from persisted state - Parallel drain: multiple worker threads can drain the same queue - Deploy-safe: rolling deployments don't drop in-flight work - Observable: queue depth and drain rate become SLIs

Canonical example

Instacart Blueberry's diagnostic passes are enqueued onto a Postgres-backed durable queue. "For an on-call system that has to keep responding at 3am during a rolling deployment, this is a load-bearing decision." (Source: sources/2026-07-14-instacart-blueberry-on-call-reasoning-harness)

When to use

  • Agent workloads that must be "always on" (on-call, alerting, customer-facing)
  • Passes that take minutes (long enough for infra disruption to be probable)
  • Multi-worker architectures where work needs to be redistributable

Relationship to durable execution

This pattern is a narrower form of concepts/durable-execution — it provides durability at the job dispatch level rather than checkpointing every step within the execution. The agent's reasoning loop itself may not be checkpoint-resumable, but the job is guaranteed to be retried from scratch by another worker.

Seen in

Last updated · 585 distilled / 1,765 read