PATTERN Cited by 1 source

SQLite-keyed stage persistence

Intent

Provide crash-tolerant, resumable persistence for multi-stage agent pipelines by writing all state to a single SQLite database keyed by a composite key — typically (run_id, repo, stage) — so any stage can resume, retry, or be pulled into a later run without redoing work.

Mechanism

1. Every stage writes findings/results immediately as they happen (streaming persistence).
2. A crash costs only the single task in flight — all prior completed tasks are durable.
3. Composite key allows selective replay: retry one stage for one repo without affecting others.
4. Natural fit for single-writer pipelines with many readers across stages.

Key caution

"Sometimes a transient API error comes back as text in the (200 OK) response stream instead of throwing a code exception. To the orchestrator, this looks exactly like a task that finished cleanly. You must explicitly classify the response text, not just trust the exception type, or you end up logging empty runs as successes." (Source: sources/2026-06-18-cloudflare-build-your-own-vulnerability-harness)

Seen in

• systems/cloudflare-vulnerability-discovery-harness — all VDH state in one SQLite DB; "persistence needs to be factored in before parallelism"

Related

• concepts/stateless-agent-compute — the principle this pattern implements
• patterns/per-repo-budget-cap — budget enforcement reads from the same persistence layer