Complete database of 84 web crawlers across 8 categories — AI bots, search engines, SEO tools, social media & more. Check how each one interacts with your robots.txt.
84Total Crawlers
8Categories
6Tiers
25+AI / LLM Bots
Browse by Category
All Bots
Bot NameCompanyTierPurposeSafety
Top Operators
Keyword & Domain Explorer
Keyword Research
Explore by seed keyword or scan an entire domain — get Google Trends data, difficulty scores, search intent, CPC, and AI citation readiness.
Fetching keyword suggestions from Google…
Generative Engine Optimization
Can AI Search Engines Actually Find You?
Instantly audit your site's GEO readiness — see exactly how ChatGPT, Perplexity & Google AI Overviews crawl, read, and cite your content.