How to show up in Perplexity and Claude answers: the technical checklist
Perplexity and Anthropic each run separate search, user and training bots. Here's which ones to allow, how firewalls get in the way, and what to fix first.
Perplexity and Claude both search the live web and cite sources. Like OpenAI, both companies split their crawlers by job, and the names matter.
Perplexity
- PerplexityBot surfaces and links websites in Perplexity's search results. Perplexity says it isn't used to crawl content for AI foundation models, and it recommends allowing it.
- Perplexity-User fetches a page when a user's question needs it. Perplexity says this fetcher generally ignores robots.txt, since a person asked for the page.
Perplexity says changes can take up to 24 hours to show up, and it publishes IP lists for both bots.
Anthropic (Claude)
- Claude-SearchBot indexes content to improve Claude's search results. Anthropic says blocking it may reduce your visibility in Claude's answers.
- Claude-User fetches pages when a Claude user asks a question. Blocking it stops Claude from reading your page on request.
- ClaudeBot collects content that may be used for training. Blocking it opts your future content out of training.
Anthropic says its bots honour robots.txt, including the non-standard Crawl-delay, and that you need rules for each subdomain. It also warns that blocking by IP address isn't a reliable opt-out, because it stops the bot from reading your robots.txt at all.
User-agent: PerplexityBot
User-agent: Claude-SearchBot
User-agent: Claude-User
Allow: /
# training opt-out (optional)
User-agent: ClaudeBot
Disallow: /The firewall problem
Perplexity's own docs include instructions for allowing its bots through Cloudflare and AWS WAF, because security rules catch them so often. If you use bot protection, add an allow rule that matches both the user agent and the published IP ranges. Matching the name alone is easy to spoof.
Then make the page worth quoting
Neither company publishes how it picks sources, so treat any "Perplexity ranking factor" list with suspicion. The safe bets are the same everywhere: clear facts in server-rendered text, one page per service, accurate structured data, and consistent details across the web.
Two checks: the AI Crawler Checker reads your robots.txt for all of these bots, and the AI Visibility Quick Scan asks an AI model real customer questions about your category and tells you whether you come up.
Sources
How to show up in each AI assistant
All engine guides →Rather not do this yourself? We'll do the whole thing and send you the receipts.