Skip to content

PATTERN Cited by 1 source

IDP extended to AI agent tools

Definition

The IDP-extended-to-AI-agent-tools pattern is the framing / architectural pattern of taking the Internal Developer Portal (IDP) shape that catalogs human-operated services — canonical instance: Backstage — and extending it "one layer up" to catalog the tools AI agents call. Same four functions (discovery + version metadata + ownership + dependency tracking); same dual-consumer surface (UI for humans, API for machines); different cataloged entities (tools, not services) and different machine consumers (agents, not deployment / monitoring tooling).

"Think of it as an Internal Developer Portal (IDP) built for the agent era, solving the same coordination problem that IDPs solved for service teams, but one layer up." — Source: sources/2026-05-11-mongodb-fighting-tool-sprawl-the-case-for-ai-tool-registries

The "one layer up" relationship

The phrase is precise and the pattern depends on it:

Layer What's cataloged Human consumer Machine consumer
IDP (Backstage) Services humans operate Engineers / SREs / on-call Deployment + observability tooling
AI tool registry Tools agents call Engineers / security reviewers / tool owners Agents at decision time

The cataloged entity changes (service → tool) and the machine consumer is now an agent running at request-time speed rather than a tooling pipeline running at deployment-time speed. The four-function shape (patterns/enterprise-ai-tool-registry: discovery + versioning + certification metadata + access control) carries directly across.

What the IDP era already proved

The pattern's argument rests on the IDP era's empirically- demonstrated lessons:

  1. Catalogs work. Organisations with N services need a service catalog regardless of how disciplined teams are. "Platform teams already understand why IDPs exist."
  2. Catalogs are infrastructure. Treating them as application-layer concerns produces sprawl + duplication + visibility gaps. The same argument now applies to AI agent tooling.
  3. Dual-surface matters. Backstage's UI surface (human browsing) and its API surface (programmatic queries from CD pipelines, monitoring, etc.) are both load-bearing. The same dual-surface property is structurally required for the AI-tool-registry use case where humans browse the catalog and agents query it.
  4. Federation is workable. Backstage demonstrates that a single catalog can hold heterogeneous entity types (services, APIs, resources, systems, domains, groups) without collapsing. AI tool registries face the same property: tools span MCP servers, REST APIs, internal RPC services, code-execution sandboxes, data accessors.

What the agent era adds that IDPs didn't have

The pattern is structural, not metaphorical — but the AI tool registry context introduces new constraints IDPs didn't face:

  1. Per-agent-identity authorization at machine speed. An agent calls dozens of tools per minute; the registry's access-control surface is consulted at machine speed, not at deployment time. IDPs are typically consulted at human-speed (filing tickets, opening dashboards) or at deployment-time speed (CI / CD).
  2. Agent-readable schema discovery. Tools must expose machine-parseable schemas (parameter types, return shapes, side effects) at request time so the agent can decide whether to call them. IDPs catalog services as artifacts, but service schemas live in OpenAPI / gRPC IDL files referenced by the catalog rather than first-class entries.
  3. Runtime certification-status checks during a tool call. The policy layer reading from the registry runs in the request path; an unreviewed tool's call must be denied during the agent's invocation, not after. IDPs typically surface certification status at adoption / deployment time, not at request time.
  4. Tool-version semantics interacting with running agents. A tool-schema change mid-conversation can break an in-flight agent; the registry's versioning needs to support runtime schema-version pinning in a way IDP service-version tracking didn't have to.

These differences mean the pattern is not just rebrand Backstage for agents — but the structural template carries forward, and that's the pattern's contribution.

Composes with

  • patterns/enterprise-ai-tool-registry — the operational pattern this framing motivates. The four-function shape (discovery + versioning + certification metadata + access control) is the IDP-pattern carryover.
  • patterns/hosted-mcp-ecosystem — Pinterest's hosted ecosystem is the in-production realisation of this framing at the MCP layer; the systems/pinterest-mcp-registry is the agent-era IDP for MCP-compliant tools, with all the Backstage-shape properties (web UI for humans, API for agents, per-server cards, ownership, certification status).
  • Service-catalog plugins for IDPs — a registry can be implemented as a Backstage plugin (or equivalent), reusing the IDP framework rather than building from scratch. The pattern doesn't prescribe the implementation choice.
  • vs patterns/enterprise-ai-tool-registry: this pattern is the framing (IDP-shape extended to agents); the enterprise-AI-tool-registry pattern is the operational shape (four functions, two pillars, organisation-internal scope). The two are sibling — one motivates the other; both describe the same architectural artifact at different levels.
  • vs patterns/hosted-mcp-ecosystem: hosted MCP ecosystem is a protocol-scoped instance of the IDP-extended pattern at the MCP layer; this pattern is the broader framing across any agent tool protocol.
  • vs public API catalog (RFC 9727): RFC 9727 is a per-service public discovery primitive; this pattern is organisation-internal multi-tool inventory. They compose at the boundary — a registered tool can publish a public API-catalog entry — but operate at different scopes.

When to use

  • Existing IDP investment — the organisation already runs Backstage or equivalent and adding agent tools as a new entity type is cheaper than greenfield.
  • Engineering culture aligned with platform-engineering patterns — IDP discipline is already familiar; the agent-era extension is a recognisable shape rather than a new concept.
  • Tool population spans heterogeneous protocols — the IDP framework's federation property (multiple entity types in one catalog) handles MCP + REST + RPC + sandboxed-code tools without requiring a separate substrate per protocol.

When NOT to use

  • No IDP investment + small agent population — building an IDP-shape catalog from scratch is over-engineering for fewer than ~10 tools.
  • Existing AI gateway already owns the inventory — a central proxy with built-in tool catalog (e.g. Unity AI Gateway on the Databricks side, AI Gateway on the Cloudflare side — see concepts/centralized-ai-governance) collapses registry + policy into one component, removing the IDP-shape extension's payoff.

Seen in

Last updated · 542 distilled / 1,571 read