What is Content-Signal in robots.txt?
A newer addition to robots.txt that separates two questions people had been forced to answer together: who is allowed to read your site, and what they are allowed to do with what they read.
What it actually is
Traditional robots.txt is a door policy — a crawler is either in or out. Content-Signal is closer to terms of use posted beside the door. It lets you say that your content may be used to answer someone's question right now, but not to train a model; or that search indexing is welcome while other uses are not. Same visitor, different permissions for different purposes.
Why it matters
Without it you face an all-or-nothing choice, and most businesses get that choice wrong in one of two expensive directions. Block everything and you disappear from AI answers to protect content nobody was going to steal. Allow everything and you hand over material you would rather have licensed. Content-Signal is how you keep the visibility while setting terms on the rest — which for most service businesses is exactly the position they actually hold.
What good looks like
A robots.txt that states your position on the distinct uses — being found and cited in answers, versus being consumed as training material — rather than leaving it to be inferred from a crawler allow-list. The point is that your stance is written down somewhere a machine can read it, instead of living only in your head.
Being honest about it
This is genuinely new and adoption is early — the operators who honour it are a minority today, and it carries no ranking benefit whatsoever. Treat it as a low-cost way of putting your position on the record rather than as enforcement. It is not a licence, it is not a contract, and anyone telling you it protects your content is overstating what it does.
Where most sites go wrong
The common failure is silence: nothing is stated, so every crawler applies its own default, and those defaults differ from company to company and change without notice. The subtler failure is contradiction — a site that blocks AI crawlers outright while separately declaring its content available as AI input. Whoever reads that has to guess which you meant, and their guess will not reliably be the generous one.