CONCEPT Cited by 1 source

Markdown content negotiation

Definition

Markdown content negotiation is the convention that when a client sends Accept: text/markdown on an HTTP request, the server responds with a clean markdown representation of the same URL instead of HTML. It's standard HTTP content negotiation per RFC 9110 applied to a new MIME type (text/markdown).

Why it matters for agents

HTML wastes tokens: navigation chrome, <div> soup, inline styles, hydration markers, ads, cookie banners, analytics scripts, related-articles widgets — none of that is content for an LLM. Markdown is ~what the LLM was trained on; {heading, paragraph, list-item, link, code-block} is all it needs.

Measured on Cloudflare's own docs (2026-04-17):

"The markdown version requires far fewer tokens — we measured up to 80 % token reduction in some cases — which makes responses faster, cheaper, and more likely to be consumed in its entirety, given the limits on context windows that most agent tools have by default."

At scale, 80 % fewer tokens per doc-fetch is 5× more calls per context window, 5× fewer dollars per agent-turn, and 5× better odds of fitting the relevant doc in a single read rather than paginating into a grep loop.

Adoption (2026-04)

• 3.9 % of the top 200 k domains handle markdown content negotiation correctly, per Cloudflare Radar.
• Only 3 of 7 agents Checkly tested (February 2026) send Accept: text/markdown by default: Claude Code, OpenCode, Cursor. Reference: State of AI Agent Content Negotiation.

These two numbers together argue for a URL-based fallback — don't assume the agent will send the header. See patterns/dynamic-index-md-fallback for Cloudflare's Transform-Rules-based /index.md approach.

Failure modes and caveats

• Caching surface increases. A CDN now caches per-Accept variant; you need Vary: Accept at minimum or a URL- differentiated path (/index.md).
• Rendering parity is load-bearing. If the markdown is a stripped-down pre-render, content can diverge from the HTML version over time unless the server renders both from one source of truth.
• Most agents don't send the header. Until agent defaults change, rely on /index.md-style URL fallbacks (patterns/dynamic-index-md-fallback) and / or hidden HTML directives that tell the LLM about the markdown URL.

Seen in

• sources/2026-04-17-cloudflare-introducing-the-agent-readiness-score-is-your-site-agent-ready — canonical wiki instance. Cloudflare's developers.cloudflare.com implements markdown content negotiation via two Transform Rules (URL Rewrite + Request Header Transform) and backs the default check in the Agent Readiness Score's Content-for-LLMs dimension. Quoted 80 % token reduction.

Related

• concepts/llms-txt — complementary primitive (curate which URLs; content negotiation serves each efficiently).
• patterns/dynamic-index-md-fallback — URL-based fallback for clients that don't send the header.
• patterns/hidden-agent-directive-in-html — complementary signal that tells LLMs reading HTML about the markdown URL.
• concepts/agent-readiness-score — where this is the default-enabled Content-for-LLMs check.
• concepts/machine-readable-documentation — parent concept; markdown-over-content-negotiation is one of the documentation-as-data primitives.
• systems/cloudflare-developer-documentation — reference implementation.