Bot directory · User-triggered fetchers

User-triggered fetchers · Anthropic

Claude-User

Claude-User is the user agent Claude uses when someone asks it a question and it needs to read a web page to answer. Unlike the user-triggered fetchers of OpenAI and Perplexity, Anthropic doesn’t exempt it from robots.txt, so a disallow in robots.txt stops it.

Checked against Anthropic’s documentation on 19 October 2026

Operator
Anthropic
Purpose
Live user request. Fetchers that load a page because a person asked an assistant about it. Blocking one means the assistant can’t open your pages on request.
robots.txt token
Claude-User
We test with
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-User/1.0; +Claude-User@anthropic.com)
a string with the Claude-User token. Anthropic doesn’t publish the full string.
Documentation
support.claude.com/en/articles/8896518

What Claude-User does

Claude-User supports Claude AI users. When individuals ask questions to Claude, it may access websites using a Claude-User agent.

support.claude.com

Disabling Claude-User on your site prevents our system from retrieving your content in response to a user query, which may reduce your site’s visibility for user-directed web search.

support.claude.com
BotWhat it doesModel training
ClaudeBotCollects web content that could contribute to training Anthropic’s modelsYes
Claude-SearchBotIndexes pages to improve Claude’s search resultsNot described as training
Claude-UserOpens a page when a Claude user asks a question that needs itNot described as training

The Claude-User user agent

Anthropic names its bots’ robots.txt tokens but doesn’t publish their full user agent strings. Match on the token, which appears in every request: Claude-User, case-insensitively. Strings quoted elsewhere are observations, not Anthropic’s documentation.

The token is unique among Anthropic’s bots: a rule matching claude-user doesn’t catch the other two.

How Claude-User reads robots.txt

Anthropic’s Bots respect “do not crawl” signals by honoring industry standard directives in robots.txt.

support.claude.com

Claude-User IP addresses and how to verify them

Anthropic publishes one list of addresses for all three of its bots, so a match tells you a request came from Anthropic, not which bot sent it; the user agent says that. Anthropic documents no reverse DNS names.

ListCoversIPv4IPv6Dated
claude.com/crawling/bots.jsonClaudeBot, Claude-SearchBot and Claude-User3807 October 2026

Anthropic recommends robots.txt over IP blocking for opting out of Claude-User: “Alternate methods like blocking IP address(es) from which Anthropic Bots operates may not work correctly or persistently guarantee an opt-out, as doing so impedes our ability to read your robots.txt file.” If you block its addresses, keep /robots.txt open to them.

Check an IP now

Paste an address from your logs. We match it against Anthropic’s current list, along with the lists of the other operators that publish them.

Block Claude-User at your firewall or CDN

robots.txt asks; a firewall enforces. Blocking Claude-User means Claude can’t read your pages when its users ask about them. Match the name case-insensitively and anywhere in the string, never a full string or a version.

Cloudflare

In Security → WAF → Custom rules, create a rule with this expression and the action Block. Cloudflare’s contains is case-sensitive, hence lower().

(lower(http.user_agent) contains "claude-user")

Cloudflare also has one-click controls: AI bot policies, which block AI crawlers by what they do (search, agent or training), and AI Crawl Control, which lets you allow or block crawlers one by one and shows which of them fetch your pages.

nginx

# in the http block
map $http_user_agent $blocked_bot {
    default 0;
    ~*claude-user 1;
}

# in the server block
if ($blocked_bot) { return 403; }

Block Claude-User

Add a group naming it to /robots.txt. Compliant crawlers stop before requesting any page. A firewall rule on the user agent is the only way to enforce it against crawlers that ignore robots.txt. Anthropic honours this for Claude-User too: Claude then can’t open your pages when someone asks about them.

User-agent: Claude-User
Disallow: /

Allow Claude-User when everything else is blocked

A group naming the bot wins over User-agent: *, so an explicit allow lets it in while your catch-all rules stay in place.

User-agent: Claude-User
Allow: /

User-agent: *
Disallow: /

Does Claude-User reach your site?

The checker applies your robots.txt the way Claude-User does, then requests your page with the user agent above, alongside 29 other bots, and opens the report on Claude-User.

Questions about Claude-User

What is Claude-User?

Claude-User is the user agent Claude uses to open a web page when a user’s question needs it. It doesn’t crawl on its own.

Does Claude-User respect robots.txt?

Yes. Anthropic says its bots honour robots.txt and doesn’t make an exception for Claude-User, unlike OpenAI’s ChatGPT-User and Perplexity’s Perplexity-User, which may not follow it.

What happens if I block Claude-User?

Anthropic says it prevents Claude retrieving your content in response to a user’s query, which may reduce your site’s visibility for user-directed web search.

What IP addresses does Claude-User use?

The addresses in Anthropic’s list at claude.com/crawling/bots.json, which it publishes for all three of its bots: 38 IPv4 ranges and no IPv6 in October 2026.

Also run by Anthropic: Claude-SearchBot, ClaudeBot.

Other user-triggered fetchers: ChatGPT-User, Perplexity-User, MistralAI-User, Meta-ExternalFetcher.

Build a complete file with the robots.txt generator.