AI Crawler Access Checker

Can ChatGPT, Claude, Perplexity and Google’s AI actually read your site? Check your robots.txt, llms.txt and firewall in one pass - free, no sign-up.

We fetch three fixed pages from your site (robots.txt, llms.txt and the homepage) with our published fetcher, RankedContentBot. Results are cached for a day and the URL is shareable.

What this tool checks

robots.txt, per AI crawler

GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended, Bingbot and CCBot - evaluated using the longest-match rules real crawlers use (RFC 9309).

llms.txt

Determines if your site includes the emerging llms.txt convention that gives AI systems a curated overview of your site.

Firewall challenges

Detects if your host, CDN or firewall answers automated readers with a challenge instead of your content - the silent way sites disappear from AI answers.