Googlebot Desktop user agent string
Handles the smaller share of Googlebot crawls that simulate a desktop user. It shares the robots.txt token Googlebot with the smartphone crawler, so robots.txt cannot target either one alone. Below is the exact string it sends, what each part means, and how to allow or block it.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/154.0.8037.92 Safari/537.36
At a glance
| Type | Search engine crawler |
| Operator | |
| Purpose | Desktop crawling for Google Search |
| Version | 2.1 |
| robots.txt token | Googlebot |
| How to verify | Reverse DNS resolves to crawl-*.googlebot.com or geo-crawl-*.geo.googlebot.com, or the IP is in Google's common-crawlers.json. |
Allow or block it in robots.txt
Block Googlebot from the whole site
User-agent: Googlebot
Disallow: /Allow it again
User-agent: Googlebot
Allow: /String breakdown
| Fragment | Meaning |
|---|---|
Mozilla/5.0 | Historical compatibility token. Nearly every browser claims "Mozilla/5.0" so that old servers do not serve it broken pages. |
AppleWebKit/537.36 | WebKit rendering engine, build 537.36. Chromium browsers keep this token for compatibility. |
KHTML, like Gecko | Compatibility note: the engine descends from KHTML and behaves like Gecko. Kept for legacy browser sniffers. |
compatible | Compatibility token — historically sent by IE, now mostly by bots. |
Googlebot/2.1 | Googlebot 2.1 — an automated crawler/bot, not a real browser. |
+http://www.google.com/bot.html | Info page the bot operator publishes for webmasters. |
Chrome/154.0.8037.92 | Chrome (or a Chromium-based browser), version 154.0.8037.92. |
Safari/537.36 | Safari/WebKit build 537.36 — a compatibility token, not the Safari version. Chromium browsers send it too. |
Use it in code
curl
curl -A "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/154.0.8037.92 Safari/537.36" https://example.comPython (requests)
import requests
requests.get("https://example.com", headers={"User-Agent": "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/154.0.8037.92 Safari/537.36"})Playwright (Node.js)
const context = await browser.newContext({ userAgent: "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/154.0.8037.92 Safari/537.36" });Source: Google's common crawlers · checked September 2026