What this crawler family is
Google publishes the cleanest example of why crawler policy should be family-aware instead of brand-aware. Google is not a sufficient answer to the operational question you need to make.
comparison
Googlebot vs Special Crawlers vs User-Triggered Fetchers
Compare Google's main crawler families so you can separate classic search crawl traffic from Google's product-specific or user-triggered fetch behavior.
Verification and source data
IPBot uses the source inventory below to decide whether a request is merely known or fully verified.
Swipe sideways to review every verification column.
| Crawler | Intent | Verification | CIDRs | Freshness |
|---|---|---|---|---|
| Googlebot
Official source
| search | Reverse + forward DNS | 311
IPv4 167 / IPv6 144 | 2026-05-22 |
| Google Special Crawlers
Official source
| special case | Known IP range | 266
IPv4 133 / IPv6 133 | 2026-05-22 |
| Google User-Triggered Fetchers
Official source
| user triggered | Known IP range | 1048
IPv4 524 / IPv6 524 | 2026-05-22 |
Whether to allow, block, or separate it
Treat Googlebot as the main search crawler, then review special crawlers and user-triggered fetchers according to the product workflow they support. Do not assume the same access decision fits all three.
robots.txt guidance
Keep a documented record of which Google family each rule or exception is meant to target. That prevents SEO rules from accidentally breaking product previews or publisher tools.
Cloudflare and WAF guidance
Use stricter verification for Googlebot itself and avoid broad Google allowlists that flatten every Google crawler family into the same outcome.
Googlebot
Standard `Googlebot` controls apply through `robots.txt` and search indexing directives.
Safe to treat as a verified crawler when reverse and forward DNS both succeed.
Google Special Crawlers
These are not the same as generic Googlebot indexing traffic; apply product-specific rules where needed.
Keep these separate from search crawler rules because the operational purpose differs.
Google User-Triggered Fetchers
These requests are initiated by a user or publisher workflow and should not be treated as standard search crawling.
Avoid blocking these with the same policy you use for autonomous indexing bots if previews matter.
FAQ
Which Google crawler family is the main search bot?
Googlebot is the primary search indexing crawler in this comparison.
Why separate user-triggered fetchers from Googlebot?
User-triggered fetchers serve different workflows and can be business-critical even when you want to keep search crawling tighter.