Do your AI content signals agree with each other?
Whether the AI preferences your site states are the same everywhere we can read them: in robots.txt, in the page's response headers and on its Markdown version. When they disagree, an AI client may follow whichever is the most permissive.
What it actually is
A site can state how its content may be used in several places, and each statement is read separately. robots.txt may carry a Content-Signal line, a response header may carry another, and a Markdown copy of the page is a third response with its own headers. This check compares what each says and flags a disagreement, such as training forbidden in one place and allowed in another.
Why it matters
You do not control which of the statements a machine reads first. If they differ, you have in effect published the most permissive of them. The contradiction is often accidental: some CDNs add a permissive default signal to Markdown responses when the site sent none, so the disagreement appears without anyone writing it.
What good looks like
The same signals in robots.txt, in the page headers and on the Markdown version, or none anywhere. A site that has not stated a preference has nothing to disagree about.
Being honest about it
This is an Advanced check and never a fail. It stays out of the headline score. It reads only the places we can fetch, so a disagreement in a place we could not reach is invisible, and a pass means we found no contradiction, not that none exists.
Where most sites go wrong
The classic case is a CDN feature that converts pages to Markdown and adds its own permissive default, quietly overriding the stricter line in robots.txt. The other is a signal set on the page but forgotten on the Markdown copy, or the reverse.
On the web
Not enough sites yet.
Not enough sites yet.
Not enough history yet.
Based on the sites Bot Appétit has scanned: ones people chose to scan, plus lists we scan for research. Not a random sample of the web.
Not sure where yours stands?
A free scan checks this and everything else in about 20 seconds — no signup. Or if it's already flagged and you'd rather it were simply handled, we do that too.
Flagged for this one? We handle it. No pitch, just a plain-English chat.
Book a call