Can AI crawlers actually reach my site?
AI access is about whether AI crawlers are allowed in at all, by your robots.txt and by whatever stands in front of your site. It matters to a machine because a crawler that is turned away at the door reads nothing, so every other check on the site stops mattering to it. Below are the 2 checks in this course, what each one looks for, and what good looks like. We can read what your rules say and see what our own signed bot is served; we can't see what each AI company's crawler is served, because a firewall can treat a verified crawler differently from us.
All 2 checks count toward your headline score.
The checks in this course
2 checksAI crawlers aren't blocked
Is your site blocking ChatGPT, Claude and Perplexity?Blocking the search-and-answer bots (GPTBot, ClaudeBot, PerplexityBot, Google-Extended…) removes you from AI answers entirely.
What good looks like The AI crawlers you want are allowed in robots.txt.
Read more about AI crawler accessAutomated visitors can reach your pages
Is my site blocking AI crawlers by accident?AI answer engines can only cite pages they can load. If your site turns away automated visitors indiscriminately, GPTBot, ClaudeBot and PerplexityBot go with them and you vanish from AI answers. Note the distinction: most bot-protection vendors allow VERIFIED crawlers while refusing unverified ones, so being refused is not by itself proof that AI crawlers are blocked.
What good looks like Your pages load for honest automated clients — AI crawlers aren't blocked at the door.
Read more about bot accessWords you'll meet here
How does your page read to a machine?
A free scan checks these 2 and everything else in about 20 seconds — no signup.
Rather have it handled? No pitch, just a plain-English chat.
Book a call