Database
There are currently 317 real crawlers registered in the database.
1 - 20 of 209 results
| Nr. | Name | Host | Rating | Updated | Info |
|---|---|---|---|---|---|
| 1 | 2IP-Bot/1.1 | http://2ip.io | Lower | 2025-12-01 20:53:32 | |
| 2 | AcademicResearchBot/1.0 | - | Lower | 2025-11-26 02:00:13 | |
| 3 | AdsBot-Google | http://www.google.com/adsbot.html | Lower | 2026-01-07 22:14:21 | |
| 4 | AdsBot-Google-Mobile | http://www.google.com/mobile/adsbot.html | Lower | 2025-05-22 04:00:09 | |
| 5 | AhrefsBot/7.0 | https://ahrefs.com/robot/ | High | 2026-01-19 22:55:19 | |
| 6 | AliyunSecBot/Aliyun | https://service.alibaba.com | Lower | 2026-01-19 21:43:15 | |
| 7 | Amazonbot/0.1 | https://developer.amazon.com/support/amazonbot | High | 2026-01-19 19:43:37 | |
| 8 | Applebot/0.1 | http://www.apple.com/go/applebot | Medium | 2026-01-19 22:39:57 | |
| 9 | Archive.org_bot | http://archive.org/details/archive.org_bot | Lower | 2025-04-18 01:35:07 | |
| 10 | AwarioBot/1.0 | https://awario.com/bots.html | High | 2025-11-21 11:15:49 | |
| 11 | BLEXBot/1.0 | http://webmeup-crawler.com/ | Lower | 2025-04-18 05:05:08 | |
| 12 | BacklinksExtendedBot | - | Lower | 2025-04-18 03:55:07 | |
| 13 | Baiduspider/2.0 | http://www.baidu.com/search/spider.html | Lower | 2026-01-19 22:44:58 | |
| 14 | Barkrowler/0.9 | https://babbar.tech/crawler | High | 2026-01-19 22:55:49 | |
| 15 | Bingbot/2.0 | http://www.bing.com/bingbot.htm | High | 2026-01-19 22:54:15 | |
| 16 | BitSightBot/1.0 | https://www.bitsight.com/ | Lower | 2026-01-07 17:36:20 | |
| 17 | BrightEdge-Crawler/1.0 | https://brightedge.com | Lower | 2025-04-18 02:00:08 | |
| 18 | Bytespider | https://bytedance.com | Medium | 2026-01-19 22:32:23 | |
| 19 | CCBot/2.0 | https://commoncrawl.org/faq/ | Medium | 2026-01-18 08:43:19 | |
| 20 | CareerScraper/1.0 | - | Lower | 2026-01-16 12:34:03 |
AdsBot-Google-Mobile
Host:
http://www.google.com/mobile/adsbot.html
Rating:
Lower
Updated(UTC):
2025-05-22 04:00:09
Amazonbot/0.1
Host:
https://developer.amazon.com/support/amazonbot
Rating:
High
Updated(UTC):
2026-01-19 19:43:37
Archive.org_bot
Host:
http://archive.org/details/archive.org_bot
Rating:
Lower
Updated(UTC):
2025-04-18 01:35:07
Baiduspider/2.0
Host:
http://www.baidu.com/search/spider.html
Rating:
Lower
Updated(UTC):
2026-01-19 22:44:58
BrightEdge-Crawler/1.0
Rating:
Lower
Updated(UTC):
2025-04-18 02:00:08
Bytespider
Rating:
Medium
Updated(UTC):
2026-01-19 22:32:23