Free Tool

AI Crawler Access Checker

Can ChatGPT, Claude, Perplexity and Gemini actually read your website? Paste your URL and find out in seconds — many brands block AI crawlers without knowing it.

We check your robots.txt — the public rulebook that tells crawlers which parts of your site they're welcome to visit. Some sites also block bots deeper in their security setup, which can't be seen from outside — so a clean result here means your rulebook lets AI in, even if we can't guarantee nothing else is in the way.

Your website URL
No signup needed. We fetch your robots.txt — nothing is stored.
Fetching your robots.txt
Reading your crawler rules
Checking 7 AI crawlers against your rules

How to unblock AI crawlers

  1. Find your robots.txt. It lives at the root of your site (yourdomain.com/robots.txt). On WordPress, edit it via your SEO plugin (Yoast/Rank Math → Tools); on Shopify it's a theme template (robots.txt.liquid); on custom sites it's a file in your web root.
  2. Remove or narrow the blocking rule. Look for the Disallow line shown in your results above, under the User-agent group that matched. If the block is under User-agent: *, don't delete it blindly — it may be protecting admin paths. Instead, add explicit allow groups for the AI crawlers:
User-agent: GPTBot
Allow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: PerplexityBot
Allow: /

User-agent: Google-Extended
Allow: /
  1. Check your security settings too. If your site uses Cloudflare, look in Security → Bots — a "Block AI bots" switch there overrides everything in robots.txt. Security plugins on WordPress and other platforms have similar switches.
  2. Check it worked. Re-run this checker after saving — your new rules take effect as soon as the file is live.

Deliberately blocking AI training bots is a legitimate choice for some publishers — the point is that it should be a decision, not a leftover default. If visibility in AI answers matters to your brand, the crawlers need to be able to read you.

Access is step one. Visibility is the game.

Crawlers reading your site doesn't mean AI engines recommend you. Visibly checks whether ChatGPT, Gemini, Perplexity and Claude actually name your brand — and tells you what to fix. Free brand scan launching soon.

Get early access →