Skip to content

SYSTEM Cited by 1 source

ChatGPT-User

ChatGPT-User is OpenAI's on-demand, user-triggered web crawler — documented alongside other OpenAI crawlers. Positive-control counterpart to Perplexity in Cloudflare's 2025-08-04 post: when Cloudflare ran the same controlled experiment against ChatGPT that caught Perplexity's stealth crawling, ChatGPT-User:

  1. Fetched robots.txt first.
  2. Honored the Disallow and stopped crawling.
  3. When the Disallow was removed but a block page was returned, still stopped.
  4. No follow-up crawl attempts from alternate user-agents or third-party bots.

This four-step behavior is what Cloudflare argues a cooperative AI crawler should look like. Combined with ChatGPT Agent signing its requests via Web Bot Auth — see systems/web-bot-auth / patterns/signed-bot-request — the OpenAI crawler posture is the reference against which the 2025-08-04 post measures Perplexity.

Seen in

Last updated · 200 distilled / 1,178 read