Skip to content

PATTERN Cited by 1 source

Well-known endpoint discovery

Pattern

Publish a protocol-known metadata resource at a fixed /.well-known/<name> path so any client — human, automation, agent — can locate it without out-of-band configuration, URL guessing, or markup parsing.

Umbrella over six agent-era instances that Cloudflare's 2026-04-17 Agent Readiness Score post surfaces together:

Endpoint Purpose Standard
/.well-known/api-catalog All public APIs + specs RFC 9727
/.well-known/mcp/server-card.json MCP server description pre-connect Draft (MCP #1649)
/.well-known/agent-skills/index.json Agent-skill document index Cloudflare RFC (agent-skills-discovery-rfc)
/.well-known/oauth-protected-resource OAuth authorization-server discovery RFC 9728
/.well-known/http-message-signatures-directory Friendly-bot public keys systems/web-bot-auth draft
/robots.txt (historical, not under /.well-known/) Crawl rules, Content Signals, sitemap pointer RFC 9309

The underlying convention is RFC 8615 (concepts/well-known-uri), authored by Cloudflare's own Mark Nottingham.

When to use

  • The resource is site-wide metadata, not content — identity, capabilities, policy, keys, catalog.
  • The client class is heterogeneous — you can't push a custom URL to each.
  • You need discovery-without-markup — the resource should be consumable by non-HTML clients (bots, CLI tools, agents with just HTTP).
  • Scale economics matter — agents scanning 200 k sites want headers + one /well-known/... GET per concern, not full- page scrapes.

Complementary to the fixed path, advertise the well-known resource via a Link: HTTP response header on any endpoint:

HTTP/1.1 200 OK
Link: </.well-known/api-catalog>; rel="api-catalog"

Any endpoint (/, /index.html, /api/v1/healthz) can advertise the catalog location, so an agent that lands mid-site can discover it without having to guess the origin's root URL structure.

Trade-offs

  • Root-path namespace pollution. Every new well-known resource adds a new entry; IANA registration gates uptake but the cluster grows.
  • Protocol vs. convention. Some well-known URIs are IETF RFCs (stable); others are draft (MCP Server Card) or vendor-proposed (Agent Skills index) — adoption risk.
  • No negotiation. The path is fixed per standard; there's no version negotiation beyond what each document's inner schema provides (e.g. the MCP Server Card's protocolVersion field).
  • Site-level scope only. Well-known URIs live at origin root; per-path or per-tenant metadata needs a different mechanism.

Adoption evidence (2026-04)

From Cloudflare Radar on the top 200 k domains:

  • robots.txt: 78 %.
  • /.well-known/api-catalog + /.well-known/mcp/server-card.json combined: < 15 sites in the entire dataset.
  • Others named (Agent Skills, OAuth Protected Resource Metadata) not individually quantified in the post; also in the early- adopter tail.

Seen in

Last updated · 200 distilled / 1,178 read