Everything we check

Answer engines read your HTML and never run your JavaScript. Content that only appears after JavaScript is content ChatGPT, Claude and Perplexity can never quote — even though Google and Microsoft Copilot can see it.

What good looks like The page's main content is already in the HTML the server sends, before any JavaScript runs.

AI engines read the HTML your server sends. A page whose HTML carries almost no readable text gives them almost nothing to quote.

What good looks like The core content is server-rendered and present in the raw HTML.

Real landmarks (header/nav/main/article) tell machines what each region is, so they parse and cite you cleanly.

What good looks like The page uses header, nav, main and article instead of generic divs.

Answer engines parse by passage, not whole page; self-contained sections are far more likely to be lifted.

What good looks like One idea per section (~200–400 words), with descriptive headings, lists and tables.

Declared authorship and entity signals build the E-E-A-T that makes AI confident enough to attribute you.

What good looks like A Person or Organization identified in schema, or an author meta/rel marker.

AI engines favour recently-updated content and use dates to judge relevance — but only a date they can read without guessing.

What good looks like A published and/or last-updated date in schema, article meta, or a time element with a datetime attribute.

A machine-readable index of your key content, at a fixed path, in plain text. Without it an AI client has to guess its way around your site from HTML built for human eyes. Cloudflare's own docs publish one and tell crawlers to fetch it to discover all available pages before exploring further — this is a signal AI clients read today.

What good looks like A valid /llms.txt (H1 + summary + sectioned links), advertised via a Link: rel=describedby header.

Serving a clean markdown version to agents that ask for it gives them noise-free content. Nice-to-have; few platforms support it.

What good looks like A clean markdown version of the page is available to agents — either on request, or at the same address with .md on the end.

Blocking the search-and-answer bots (GPTBot, ClaudeBot, PerplexityBot, Google-Extended…) removes you from AI answers entirely.

What good looks like The AI crawlers you want are allowed in robots.txt.

AI answer engines can only cite pages they can load. If your site turns away automated visitors indiscriminately, GPTBot, ClaudeBot and PerplexityBot go with them and you vanish from AI answers. Note the distinction: most bot-protection vendors allow VERIFIED crawlers while refusing unverified ones, so being refused is not by itself proof that AI crawlers are blocked.

What good looks like Your pages load for honest automated clients — AI crawlers aren't blocked at the door.

A deliberate allow/deny per bot (separating training bots from search bots from user-fetchers) reads as maturity and avoids accidental blocks.

What good looks like An explicit allow/disallow per AI crawler that matches your actual intent.

Content-Signal directives (search / ai-input / ai-train) let you state how your content may be used by AI.

What good looks like Clear Content-Signal directives that reflect your policy.

Link: rel=describedby / sitemap headers help agents discover your llms.txt and sitemap without parsing HTML.

What good looks like A Link header advertising your sitemap and (if present) llms.txt.

Machine-readable markup makes your facts eligible for rich results and easier for AI to quote accurately.

What good looks like At least one valid JSON-LD block describing the page.

Broken or incomplete schema is ignored — or worse, distrusted. (We show the snippet + any errors below.)

What good looks like Every JSON-LD block parses cleanly and each entity declares its @context and @type.

A question paired with a direct answer is exactly what an AI answer engine or a search result quotes. FAQPage and QAPage markup label your Q&A content in a machine-readable format that Google and the AI/search crawlers (Bing, Perplexity, RAG crawlers) still parse. Google removed the visual FAQ rich result from search listings in May 2026, but that was only the SERP feature — the markup stays valid, is still read, and the Q&A content itself is what gets cited, so keep your answers genuine and visible on the page.

What good looks like A JSON-LD FAQPage block (multiple questions, your own answers) backing a real, visible FAQ — or, for a page built around one user-answered question, a QAPage block.

sameAs links let engines confirm you're one real entity across sources — the foundation of being named by AI.

What good looks like An Organization entity — JSON-LD or microdata — carrying a logo and sameAs links to your profiles.

Type-appropriate markup (Article, Product, LocalBusiness, Breadcrumb…) unlocks the matching rich results.

What good looks like Schema types that match the page's purpose, with required fields filled.

BreadcrumbList JSON-LD earns a distinct breadcrumb rich result in search and gives AI engines a clear sense of where a page sits in your site.

What good looks like A valid BreadcrumbList on interior pages, matching the on-page breadcrumb trail.

Each redirect hop is another request an AI client or crawler has to make before it reaches your content, and some give up. A single clean redirect is normal; a chain of them is latency you are adding to every visit. A page that does not answer at all is reported separately, as a refusal rather than a score.

What good looks like The URL answers directly, or through one clean redirect.

A Cache-Control, ETag, or Last-Modified header lets browsers and CDNs skip re-downloading an unchanged page — faster repeat visits, less server load, and a small hand to crawlers hitting you often.

What good looks like The page sends a Cache-Control, ETag, Last-Modified, or Expires header.

Serving the same content on both www and non-www (or http and https) splits ranking authority.

What good looks like All variants 301 to one preferred host.

A stray Disallow: / can hide your whole site from search; a missing robots.txt leaves crawling ambiguous.

What good looks like A valid robots.txt that allows your public pages and links to the sitemap.

A meta robots or X-Robots-Tag noindex removes the page from search entirely — often left on by mistake after launch.

What good looks like No noindex on pages you want found.

The canonical consolidates ranking signals onto the URL you want and resolves duplicate-content ambiguity. It counts whether you declare it as a link element or an HTTP Link header — Google treats both the same.

What good looks like A self-referencing canonical pointing to the preferred, on-domain URL — as a <link rel="canonical"> element or a Link: rel="canonical" response header.

Without a viewport tag a phone renders the page at a pretend desktop width and shrinks the result to fit, so none of your responsive breakpoints ever fire and the page arrives as an unusable miniature.

What good looks like A width=device-width viewport meta tag.

The title is the strongest on-page relevance signal and the first thing a search engine or an answer engine reads to know what a page is. A page with no title is unnamed to a machine.

What good looks like Every page has a unique, keyword-led title element.

It doesn't rank you directly, but it owns the search snippet and lifts click-through.

What good looks like A unique description roughly 70–160 characters — ~120–160 is the sweet spot for a full snippet.

One clear H1 states the page topic to crawlers and AI; none or several muddies the signal.

What good looks like A single H1 that states the page's topic.

A clean H1→H2→H3 structure is how crawlers, screen readers, and AI parse your content.

What good looks like Headings nest in order with no skipped levels.

When the title, H1, URL, and opening line agree on the topic, relevance is unambiguous.

What good looks like The main topic appears in the title, the H1, the slug, and the first paragraph.

Alt text powers image search, gives screen-reader context, and helps machines understand the page.

What good looks like Meaningful images carry concise, descriptive alt text.

OG tags control the preview card when your links are shared or cited — a growing discovery surface.

What good looks like og:title, og:description, og:image and og:url are all set, declared with property=.

A sitemap speeds discovery and indexing, especially for newer pages with few inbound links.

What good looks like A valid /sitemap.xml, referenced from robots.txt, whose URLs resolve.

A soft-404 (a 200 for a missing page) confuses crawlers and can get junk indexed.

What good looks like Unknown URLs return a true 404 with a helpful page.

A lang attribute and UTF-8 charset help engines, browsers and screen readers render correctly.

What good looks like A valid <!doctype html>, lang attribute, and UTF-8 charset.

hreflang serves the right language/region version and prevents duplicate-content confusion.

What good looks like Valid, reciprocal hreflang tags across your language variants.

Your favicon shows next to your result in search and in some AI answer interfaces — a small but real trust and brand signal.

What good looks like A <link rel="icon"> (or apple-touch-icon) pointing at a real icon.

A client-side <meta http-equiv="refresh"> redirect is slower than a server 301, can be ignored by crawlers, and is treated as a mild cloaking smell.

What good looks like Redirects use a server 301/302, not a meta refresh.

Dead links waste crawl budget, leak authority, and frustrate readers.

What good looks like On-page links resolve (no 4xx/5xx).

Linking through redirects (or chains) wastes crawl budget and slightly dilutes authority.

What good looks like Internal links point straight at final URLs.

Descriptive anchors tell engines (and readers) what's on the other end; 'click here' tells them nothing.

What good looks like Meaningful anchor text instead of 'click here' / bare URLs.

A reasonable internal-linking structure spreads authority; excessive or all-nofollow outbound links are a smell.

What good looks like Healthy internal linking and a sensible nofollow ratio.

HTTPS is a baseline trust and ranking signal; browsers warn users away from insecure pages, and AI engines skip them.

What good looks like Every page loads over HTTPS with a valid, unexpired certificate.

If the insecure http:// version still loads, you split trust and risk duplicate content.

What good looks like http:// requests 301-redirect to the https:// version.

Browsers outright block insecure scripts, styles and frames on an https page — breaking the page — and auto-upgrade insecure images/media, which then break if the host doesn't serve https.

What good looks like Every script, style, frame, image and media file is referenced over https.

Headers like HSTS, X-Content-Type-Options and a referrer policy harden the site and are a recognised best-practice signal.

What good looks like HSTS, X-Content-Type-Options, X-Frame-Options/CSP frame-ancestors, and a Referrer-Policy are present.

A site flagged by Google Safe Browsing is hidden behind warnings and won't be recommended.

What good looks like The site is clean in Google Safe Browsing.

A /.well-known/security.txt (RFC 9116) gives security researchers a standard way to report a vulnerability — a maturity signal shipped by Google, GitHub and Meta.

What good looks like A /.well-known/security.txt with the two fields RFC 9116 requires: Contact: and a future Expires: date.

Advertises an MCP server so agents can discover and use your tools. Only relevant if you expose agent-facing infrastructure.

What good looks like A valid MCP Server Card. The current proposal (SEP-2127, still a draft) puts it at /.well-known/mcp-server-card; sites in the wild also serve /.well-known/mcp.json and /.well-known/mcp/server-card.json, and any of the three counts.

A single index of the AI-facing things you publish — MCP servers, agent cards, skills, datasets — so an agent finds them in one hop instead of guessing at paths. Adoption is early and the specification is still a draft, so not having one is normal today rather than a defect.

What good looks like A valid /.well-known/ai-catalog.json that actually lists something. An index with nothing in it is discoverable but tells an agent no more than having none.

A discoverable manifest of skills an agent can use on your site.

What good looks like A valid Agent Skills index at /.well-known/agent-skills/index.json.

A machine-readable index of your APIs (RFC 9727) lets agents find and use them.

What good looks like A valid /.well-known/api-catalog.

Discovery metadata lets agents learn how to authenticate against protected resources.

What good looks like Valid OAuth authorization-server metadata at the well-known path.

RFC 9728 protected-resource metadata is how an MCP server tells an agent which authorization server guards it — the entry point of the MCP auth handshake.

What good looks like A valid /.well-known/oauth-protected-resource pointing at your authorization server.

OIDC discovery metadata lets agents and clients learn your token, authorization and JWKS endpoints without hard-coding them.

What good looks like Valid OpenID Connect discovery metadata at /.well-known/openid-configuration.

Web Bot Auth (IETF draft) lets a BOT OPERATOR cryptographically prove its traffic: the operator publishes signing keys on its own domain, and websites verify them. This one only applies if this domain operates a bot or agent — a normal publisher site has nothing to publish here, and that's expected.

What good looks like Only if you operate a bot/agent: a JWKS at /.well-known/http-message-signatures-directory on the domain your bot signs as (e.g. chatgpt.com publishes one).

UCP (Google) is how an AI shopping agent finds out what you sell without a human browsing your site. The agent appends /.well-known/ucp to your domain and reads a JSON manifest of your services, capabilities, payment handlers and API endpoints. Only relevant if you sell something — we don't check it otherwise.

What good looks like A JSON manifest at /.well-known/ucp declaring what you sell, how to transact, and which transports you support.

OAuth tells an agent how to authenticate. auth.md tells it how to register in the first place — the step before that. Without one, an AI agent acting for a customer has no documented way to get credentials for your service.

What good looks like A /auth.md describing your registration flows, scopes and endpoints, alongside your OAuth protected-resource metadata.

DNS-based AI-discovery records help agents find your endpoints without fetching the page.

What good looks like AI-discovery DNS records are published.

Headers that reveal exact server/framework versions hand attackers a head start.

What good looks like Server and X-Powered-By headers are absent or generic.

A title outside roughly 30-60 characters either gets truncated in a search result or is too thin to say what the page is. It's a presentation detail rather than a machine-readability one, which is why it's weighted lightly — the title existing is what actually matters.

What good looks like A title of roughly 30-60 characters — long enough to be specific, short enough to show in full.

Without a card + image, links render as a small, less clickable preview on X.

What good looks like twitter:card, title, description and image are set.

Heavy, unsized images are the most common cause of slow loads and layout shift.

What good looks like Modern formats (WebP/AVIF), lazy-loading below the fold, and explicit width/height.

You can't improve what you don't measure; no analytics means no baseline and no lead attribution.

What good looks like GA4 (or a privacy-first equivalent) is installed, ideally via GTM.

Verifying the site is your direct line to index status, errors and search performance.

What good looks like Google Search Console (and ideally Bing) verification is in place.