What does the robots meta tag actually do?
It is a per-page instruction telling search engines whether they may list this page and follow its links. It is also the single most destructive line on the web when wrong, because a page carrying the wrong value disappears from search completely and silently.
What it actually is
Where robots.txt is a site-wide door policy, this is a note attached to one page. It answers two questions: may this page appear in results, and should the links on it be followed onward. A page can be perfectly crawlable, beautifully written and entirely absent from search because of this one line.
Why it matters
Almost every other SEO problem costs you gradually — a slightly worse position, a few fewer clicks. This one is binary and total. A page told not to be indexed will not rank for anything, ever, no matter how good it is or how many links point at it. And because the page still loads perfectly for you, nothing about the site looks broken. People spend months optimising pages that were switched off the entire time.
What good looks like
Pages you want found carrying either no instruction at all or an explicitly permissive one, and only the pages you genuinely want hidden — thank-you pages, internal search results, staging areas — carrying a restriction. The rule is that the instruction should match your intention, and someone should be able to say what that intention was.
Being honest about it
This is one of the few checks where we would drop everything. There is no scenario where an unintended restriction on a commercial page is acceptable, and no gradual version of the damage. If a scan flags this on a page that matters, it is the first thing to look at and everything else can wait.
Where most sites go wrong
The classic is a launch that never got undone: a site built with search engines blocked so the unfinished version stays private, then made live without that being reversed. Months later the traffic has never arrived and nobody connects the two. The other frequent cause is a template applying one page's intention to a whole section, so a single deliberate exclusion silently takes dozens of pages with it.