BotAppetitBot
BotAppetitBot is the scanner behind Bot Appétit, a free tool byMilk Moon Studio that checks whether a page can be read by AI answer engines and search crawlers.
It is not a crawler
BotAppetitBot does not roam the web, keeps no index, and never discovers URLs on its own. It fetches only when a person types an address into Bot Appétit and presses scan — one page, on demand — and then it stops. It never follows links to scan further pages.
User agent
BotAppetitBot/1.0 (+scan.milkmoonstudio.com; on-demand)How to verify a request really is us
Anyone can copy a user-agent string, so we sign our requests. Every request BotAppetitBot makes carries an RFC 9421 HTTP Message Signature (Web Bot Auth): an Ed25519 signature tagged web-bot-auth, covering the request authority, with our public key published at:
https://scan.milkmoonstudio.com/.well-known/http-message-signatures-directoryThat directory is itself signed, so nobody can mirror it and claim to be us. If a request's signature verifies against that key, it is genuinely ours. If it does not, it is not us — whatever the user-agent says.
Exactly what one scan fetches
For a single scan of one page, in one burst, we request:
- the page itself (one retry if the connection fails outright)
/robots.txt,/sitemap.xml,/llms.txt,/.well-known/security.txt,/favicon.ico- three agent-discovery files under
/.well-known/ - one deliberately non-existent path, to see whether missing pages return a real 404
- the
www/apex variant of the host, once, to check canonicalisation - up to 12 links from the page, to check they resolve — we read the status code and discard the body unread
That is roughly 25 requests, once, for a scan a person explicitly asked for. We do not repeat, poll, or return on a schedule. If the same person scans the same page again, that is another single burst.
robots.txt
We read your robots.txt and report on what it says — it is one of the things we check for you. Because a scan is a one-off fetch of a page a person asked us to look at, we treat it as a user-directed request rather than crawling. If you would rather we did not fetch your site at all, block us and we will stop.
How to block us
In robots.txt:
User-agent: BotAppetitBot
Disallow: /Or block the signature at your edge. We would rather you did not — but a block is a legitimate answer, and we will not work around one.
Contact
Something wrong, or want us to stop?jakes@milkmoonstudio.com — a real person reads it.