Do you let AI crawlers in?
We read the robots.txt file on your domain and tell you whether the crawlers behind ChatGPT, Claude, Gemini and Perplexity are allowed to read your site. No account, no email.
Who is who
There is more than one crawler per assistant — OpenAI and Anthropic each run several. And they are not equal: some decide whether an assistant will show you at all, others do not.
Fetches pages for Claude while it is looking for an answer. Close it off and Claude will not cite you.
Fetches one specific page when someone in a Claude conversation asks for it. Part of the same gateway as Claude-SearchBot.
This has no effect on ordinary search. It only controls whether Gemini may use your site, so closing it will not hurt your search rankings.
The ordinary search crawler. Gemini builds on Google's index, so blocking it keeps you out of Gemini's answers too.
This is the one behind the links ChatGPT shows when searching. Close it off and ChatGPT will not cite you in its answers.
The crawler for Perplexity, which almost always answers with links to its sources.
The Bing crawler. ChatGPT searches through Bing's index, so blocking bingbot quietly costs you citations there too.
Collects text to train the Claude model. It has nothing to do with live search: close it off and Claude can still cite you.
Collects text that ChatGPT learns from. It has nothing to do with what ChatGPT shows when someone searches right now.
What if you are blocking someone
It need not be a mistake. Plenty of sites close off the cart, the admin area or search results, and that is fine. Those pages have no place in an assistant's answer anyway.
One rule that shuts everything to everyone. Usually a leftover from the time the site was still being built, which nobody removed.
Someone put that one there on purpose. It is worth knowing whether that intent still holds.
This tool asks AI assistants nothing. It only reads one file on your site. A real visibility measurement is a different thing.