Menu

Stack Exchange (stackexchange.com): AI Crawler Rules in robots.txt

Which AI bots does Stack Exchange block in robots.txt? Not recorded on . Unknown Source · Methodology

Too few observations yet: we need 2 checks of the main question on 2 different days.

To publish rules like these for your own site, see How to Block GPTBot and Other AI Crawlers in robots.txt. A host or CDN can also block AI crawlers on its own, which robots.txt does not show: our hosting checker names the provider behind any domain, and our hosting price tracker shows what the plans we check cost.

What robots.txt says

0 of 17 bots disallowed on /

robots.txt disallows 0 of 17 monitored bots on path /. 17 not evaluated.

  • Amazonbot · Unknown
  • anthropic-ai · Unknown
  • Applebot-Extended · Unknown
  • Bytespider · Unknown
  • CCBot · Unknown
  • ChatGPT-User · Unknown
  • Claude-SearchBot · Unknown
  • Claude-User · Unknown
  • ClaudeBot · Unknown
  • cohere-ai · Unknown
  • DuckAssistBot · Unknown
  • Google-Extended · Unknown
  • GPTBot · Unknown
  • meta-externalagent · Unknown
  • OAI-SearchBot · Unknown
  • Perplexity-User · Unknown
  • PerplexityBot · Unknown

Homepage and robots.txt responses at our latest check

homepage HTTP 200 · not reachable · robots.txt HTTP 418
Homepage
HTTP 200 · not reachable
Challenge page
Detected (Cloudflare): Cloudflare: challenge marker in body
robots.txt
HTTP 418

History and how we checked

24 records since Sep 28, 2026

How we got this number

Record history 24 records for 1 question
Last record
Shown in
Switzerland
TimeValueStatus
Oct 18, 2026UnknownUnknown
Oct 17, 2026UnknownUnknown
Oct 16, 2026UnknownUnknown
Oct 15, 2026UnknownUnknown
Oct 14, 2026UnknownUnknown
Oct 13, 2026UnknownUnknown
Oct 12, 2026UnknownUnknown
Oct 11, 2026UnknownUnknown
Oct 10, 2026UnknownUnknown
Oct 9, 2026UnknownUnknown
Oct 8, 2026UnknownUnknown
Oct 7, 2026UnknownUnknown
Oct 6, 2026UnknownUnknown
Oct 5, 2026UnknownUnknown
Oct 4, 2026UnknownUnknown
Oct 3, 2026UnknownUnknown
Oct 2, 2026UnknownUnknown
Oct 1, 2026UnknownUnknown
Sep 30, 2026UnknownUnknown
Sep 29, 2026UnknownUnknown
Sep 28, 2026UnknownUnknown
Sep 28, 2026UnknownUnknown
Sep 28, 2026UnknownUnknown
Sep 28, 2026UnknownUnknown
Download JSON Download CSV