Database
There are currently 332 real crawlers registered in the database.
| Nr. | Name | Host | Rating | Updated | Info |
|---|---|---|---|---|---|
| 21 | CCBot/2.0 | https://commoncrawl.org/faq/ | Medium | 2026-02-08 14:23:40 | |
| 22 | CareerScraper/1.0 | - | Lower | 2026-01-16 12:34:03 | |
| 23 | ChatGPT Atlas/20251222192530000 | https://chatgpt.com/de-DE/atlas/ | Lower | 2026-01-11 12:25:05 | |
| 24 | ChatGPT-User/1.0 | https://openai.com/bot | High | 2026-02-09 13:34:50 | |
| 25 | CheckMarkNetwork/1.0 | http://www.checkmarknetwork.com/spider.html | Lower | 2025-12-10 15:35:14 | |
| 26 | Claude-SearchBot/1.0 | https://anthropic.com | Lower | 2025-04-18 08:40:08 | |
| 27 | ClaudeBot/1.0 | https://anthropic.com | High | 2026-02-09 02:45:48 | |
| 28 | Coccocbot-web/1.0 | http://help.coccoc.com/searchengine | Lower | 2025-05-28 22:18:44 | |
| 29 | CodaBot/1.0 | - | Lower | 2025-10-08 02:47:11 | |
| 30 | Companyspotter - 2.0.0.0 | https://companyspotter.com | Lower | 2025-11-19 11:31:12 | |
| 31 | Comscore.com/Web-Crawler | https://www.comscore.com/Web-Crawler | Lower | 2025-04-18 05:55:08 | |
| 32 | Creditreform-ESG-Bot | - | Lower | 2026-01-28 14:00:12 | |
| 33 | DATABASE | http://78.29.51.27/antivirus/?s1=CU_sloboda-shop.de | Lower | 2026-02-07 11:00:07 | |
| 34 | DENIC-Crawler/42 | https://www.denic.de/crawler | Lower | 2026-01-17 21:19:45 | |
| 35 | DataForSeoBot/1.0 | https://dataforseo.com/dataforseo-bot | High | 2026-02-09 13:40:54 | |
| 36 | DataHub | https://www.hubspot.com | Lower | 2025-08-09 19:00:08 | |
| 37 | Dataprovider | https://Dataprovider.com | Lower | 2026-02-07 04:14:25 | |
| 38 | Developers-facebook-crawler | https://developers.facebook.com/docs/sharing/webmasters/crawler | Lower | 2025-08-30 21:38:35 | |
| 39 | Developers.snap | https://developers.snap.com/robots | Lower | 2025-12-27 13:26:22 | |
| 40 | Discordbot/2.0 | https://discordapp.com | Lower | 2026-01-14 09:45:04 |
ChatGPT Atlas/20251222192530000
Host:
https://chatgpt.com/de-DE/atlas/
Rating:
Lower
Updated(UTC):
2026-01-11 12:25:05
ChatGPT-User/1.0
Rating:
High
Updated(UTC):
2026-02-09 13:34:50
CheckMarkNetwork/1.0
Host:
http://www.checkmarknetwork.com/spider.html
Rating:
Lower
Updated(UTC):
2025-12-10 15:35:14
Claude-SearchBot/1.0
Rating:
Lower
Updated(UTC):
2025-04-18 08:40:08
ClaudeBot/1.0
Rating:
High
Updated(UTC):
2026-02-09 02:45:48
Coccocbot-web/1.0
Host:
http://help.coccoc.com/searchengine
Rating:
Lower
Updated(UTC):
2025-05-28 22:18:44
Companyspotter - 2.0.0.0
Host:
https://companyspotter.com
Rating:
Lower
Updated(UTC):
2025-11-19 11:31:12
Comscore.com/Web-Crawler
Host:
https://www.comscore.com/Web-Crawler
Rating:
Lower
Updated(UTC):
2025-04-18 05:55:08
DATABASE
Host:
http://78.29.51.27/antivirus/?s1=CU_sloboda-shop.de
Rating:
Lower
Updated(UTC):
2026-02-07 11:00:07
DataForSeoBot/1.0
Host:
https://dataforseo.com/dataforseo-bot
Rating:
High
Updated(UTC):
2026-02-09 13:40:54
DataHub
Rating:
Lower
Updated(UTC):
2025-08-09 19:00:08
Dataprovider
Rating:
Lower
Updated(UTC):
2026-02-07 04:14:25
Developers-facebook-crawler
Host:
https://developers.facebook.com/docs/sharing/webmasters/crawler
Rating:
Lower
Updated(UTC):
2025-08-30 21:38:35
Developers.snap
Host:
https://developers.snap.com/robots
Rating:
Lower
Updated(UTC):
2025-12-27 13:26:22
Discordbot/2.0
Rating:
Lower
Updated(UTC):
2026-01-14 09:45:04