AI Bot Control Center

Decide which AIs get to learn from you. Block the scrapers (training data) while keeping the search traffic (referrals).

GPTBot

OpenAI (ChatGPT Search)

CCBot

Common Crawl (Training Data)

Google-Extended

Gemini (Training Data)

ClaudeBot

Anthropic (Claude)

Applebot-Extended

Apple Intelligence

PerplexityBot

Perplexity AI

robots.txt

1User-agent: *
2Allow: /
3
4# GPTBot - OpenAI (ChatGPT Search)
5User-agent: GPTBot
6Allow: /
7
8# CCBot - Common Crawl (Training Data)
9User-agent: CCBot
10Disallow: /
11
12# Google-Extended - Gemini (Training Data)
13User-agent: Google-Extended
14Disallow: /
15
16# ClaudeBot - Anthropic (Claude)
17User-agent: ClaudeBot
18Allow: /
19
20# Applebot-Extended - Apple Intelligence
21User-agent: Applebot-Extended
22Disallow: /
23
24# PerplexityBot - Perplexity AI
25User-agent: PerplexityBot
26Allow: /
27
28Sitemap: https://yourwebsite.com/sitemap.xml

Why block training bots?

Bots like CCBot and Google-Extended are primarily used to train models, not to drive traffic. Blocking them prevents your content from being absorbed into the "black box" without attribution.