Skip to content

PATTERN Cited by 1 source

AI-powered admin agent (self-debug)

Pattern

Run a second agent whose job is operational introspection of the first. Give the second agent a read-only tool surface over the primary agent's telemetry (usage stats, error logs, request transcripts, ad-hoc SQL, charts). Operators ask the admin agent questions in natural language; it uses its tools to answer.

The canonical Vercel formulation: "You debug your agent with an agent."

Canonical Vercel framing

From the Knowledge Agent Template launch:

"There's also an AI-powered admin agent. You can ask it questions like: 'what errors occurred in the last 24 hours', or 'what are the common questions users ask'. It will use internal tools (query_stats, query_errors, run_sql, and chart) to provide answers directly. You debug your agent with an agent."

(Source: sources/2026-04-21-vercel-build-knowledge-agents-without-embeddings)

The four disclosed admin tools

  • query_stats — summary metrics (usage, errors, latency distributions, per-platform breakdowns).
  • query_errors — error-log surface; time- windowed, typed.
  • run_sql — escape hatch for ad-hoc queries against the admin DB.
  • chart — visualisation primitive for trends.

All four are read-only by design. The admin agent doesn't mutate the primary agent's configuration; it reads telemetry.

Why this is the right shape

Three pulls toward this architecture:

  • Operator's tool is the same as the operator's subject. If the team already builds agent infrastructure, building a second agent for introspection reuses the same libraries, same SDKs, same prompts, same CI.
  • Natural-language interface suits ad-hoc questions. Operators' questions are under-specified ("what's going wrong lately?") and shift every debugging session. Pre-built dashboards answer yesterday's questions; an agent answers today's.
  • Debugging-with-an-agent gives trace reuse. If you're already generating retrieval traces for the primary agent (per concepts/traceability-of-retrieval), the admin agent can read those traces as another source.

The dual to primary-agent retrieval

Axis Primary agent Admin agent
Corpus Knowledge snapshot Telemetry + logs + request history
Tools bash / bash_batch query_stats, query_errors, run_sql, chart
Users End-user across platforms Operator
Access Read-only Read-only
Pipeline Same AI SDK pipeline Same AI SDK pipeline

Two agents sharing a pipeline — one faces the user, one faces the operator. This is the meta-introspection composition that comes for free once you have one agent pipeline.

Trade-offs

  • run_sql is strong. Schema-scoped, role- permitted, rate-limited read-only SQL is a bounded risk. Write-SQL leakage through prompt injection is a real concern — the post doesn't walk the permissions model.
  • Admin-agent hallucinations are operator-facing. If the admin agent hallucinates "no errors in the last 24h" when there were errors, the operator acts on a false signal. Typed tool outputs and citations back to raw data are the mitigation.
  • Second-agent cost. You're running two agents; double the model-call cost. In return, you get operational ergonomics most teams don't ship.
  • Access-control surface grows. The admin agent needs scoped access to telemetry most of the system doesn't; misconfiguration leaks production data into operator prompts.
  • Prompt-injection vector. User-generated content captured in logs can leak into the admin agent's context when an operator runs "what did users say today?" The usual injection-safety hygiene applies.

Adjacent patterns

Seen in

Last updated · 476 distilled / 1,218 read