Robots Meta Tag
The robots meta tag is a <meta name="robots"> tag placed in the HTML <head> that tells search engines, on a page-by-page basis, how to index the page and how to display it in search results. It is the meta tag that carries individual directives such as noindex, nofollow, and max-snippet — and the umbrella term for that whole family of page-level rules.
The robots meta tag is a <meta name="robots"> tag placed in the HTML <head> that tells search engines, on a page-by-page basis, how to index the page and how to display it in search results. It is the meta tag that carries individual directives such as noindex, nofollow, and max-snippet — and the umbrella term for that whole family of page-level rules.
Why It Matters
While robots.txt controls crawler access at the site level, the robots meta tag gives you precise, per-page control over indexing and result appearance. It covers both "should this page be indexed at all" — keeping login or thank-you pages out of results with noindex — and "how should it look" — capping snippet length with max-snippet. That dual role makes it one of the most frequently used tools in technical SEO.
Key Directives
These are the main directives Google supports:
| Directive | Meaning |
|---|---|
all | No restrictions (the default) |
noindex | Exclude the page from search results |
nofollow | Do not follow links on the page |
none | Equivalent to noindex, nofollow |
nosnippet | Show no text snippet or video preview |
max-snippet:[number] | Cap the text snippet at a character count |
max-image-preview:[setting] | Set image preview size (none, standard, large) |
max-video-preview:[number] | Cap video previews at a number of seconds |
noimageindex | Do not index images on the page |
notranslate | Do not offer translation in results |
indexifembedded | Allow indexing when embedded via iframe despite noindex |
unavailable_after:[date] | Drop the page from results after a date |
Multiple directives can be combined with commas:
<meta name="robots" content="noindex, nofollow">
<meta name="robots" content="max-snippet:120, max-image-preview:large">
How to Implement It
name="robots" applies to all crawlers, while a specific user agent such as name="googlebot" targets one crawler. When rules conflict, the more restrictive rule wins — a page with both max-snippet:50 and nosnippet gets nosnippet.
For non-HTML files like PDFs and images where a meta tag cannot be inserted, the same directives are delivered through the X-Robots-Tag HTTP response header.
The most common mistake is mixing it with a robots.txt block. If crawling is blocked in robots.txt, search engines cannot open the page and therefore never see the robots meta tag — so a noindex directive gets ignored. Any page you want noindexed must remain crawlable.
Snippet Controls in the AI Era
nosnippet and max-snippet were originally about trimming result snippets, but their meaning has expanded with AI search. Google's documentation now states that these controls apply to web search, Images, and Discover as well as AI Overviews and AI Mode. Declaring nosnippet also blocks the content from being used as direct input for AI answers, and max-snippet caps how much of it can be quoted.
In other words, the robots meta tag has gone from "how many lines show in the SERP" to "how much of my content AI answers may use." Keep the tradeoff in mind: the tighter the restriction, the less visible your content becomes in AI-driven search experiences.
Sources:
- Robots Meta Tags Specifications - Google Search Central
- AI Features and Your Website - Google Search Central
How inblog Helps
inblog manages robots meta tags automatically, so regular posts need no extra setup. If a post should stay out of search results, use the per-post noindex setting. For most blogs the defaults are the right choice for the remaining directives — and if growing AI search visibility is the goal, leaving snippets unrestricted works in your favor.