CONCEPT Cited by 1 source

On-call automation

Definition

On-call automation is the use of automated systems (increasingly AI agents) to compress the time between an alert firing and the on-call engineer having actionable understanding. The core insight from Instacart's Blueberry system: "Most on-call pain does not start with a major incident. It starts in the noisy window right after a page fires, when the thread fills with links, dashboards, guesses, and partial context."

The automation targets two specific latencies: - concepts/time-to-first-insight (TTFI) — how fast the team gets a grounded first explanation - concepts/time-to-test-theories (TTTT) — how fast proposed causes can be validated against evidence

Key properties

1. Operates in the same communication channel where engineers already coordinate (Slack thread)
2. Gathers evidence in parallel rather than sequentially — historical context + live signals + change history
3. Produces grounded explanations backed by specific logs, traces, deploys — not generic suggestions
4. Supports multi-turn investigation where engineers can direct deeper dives

Seen in

• sources/2026-07-14-instacart-blueberry-on-call-reasoning-harness — Blueberry: ~17k auto-triages + ~7k engineer-initiated follow-ups monthly, ~3 min average for both TTFI and TTTT