Should your site serve Markdown to AI crawlers?
If you can, yes. A growing number of sites will hand an AI assistant a clean Markdown version of a page instead of the full HTML — less noise for the machine, and far less room for it to misread you.
What it actually is
When something requests a page, it can say what format it would prefer. A browser wants HTML. An AI assistant would much rather have the words. Markdown negotiation means the site notices that preference and answers with a stripped-back version: the headings, the paragraphs, the links, and none of the navigation, cookie banners, tracking scripts or layout scaffolding wrapped around them.
Why it matters
An assistant reading raw HTML has to work out which parts of the page are the content and which are furniture, and it does not always get that right. Serving Markdown removes the guesswork. In practice that means fewer summaries that quote your cookie notice, fewer that miss the point of the page, and a better chance the assistant repeats what you actually said rather than what it inferred from the wreckage.
What good looks like
A site that, when asked for a Markdown representation, returns one — with the real page content intact and the surrounding chrome gone. The Markdown should say the same thing the page says. A version that quietly drops half the article is worse than not offering one, because now the assistant is confidently working from a partial copy.
Being honest about it
This is an emerging capability rather than a settled standard, and most sites do not offer it. It is not a ranking factor and no engine requires it. On some platforms it is a genuine piece of infrastructure work rather than a setting. We flag it because the sites that do offer it get read more accurately, not because its absence is a problem you must solve this quarter.
Where most sites go wrong
The most common issue is not absence but a mismatch between what the platform advertises and what actually reaches a crawler. A layer in front of the site — a proxy, a CDN, a cache — can strip or ignore the preference, so the feature looks enabled from the inside and does nothing from the outside. The only way to know is to ask the way a crawler asks, from outside your own network, which is exactly what a scan does.