Database
There are currently 283 real crawlers registered in the database.
1 - 20 of 178 results
| Nr. | Name | Host | Rating | Updated | Info |
|---|---|---|---|---|---|
| 1 | AdsBot-Google | http://www.google.com/adsbot.html | Lower | 2025-11-07 13:31:25 | |
| 2 | AdsBot-Google-Mobile | http://www.google.com/mobile/adsbot.html | Lower | 2025-05-22 04:00:09 | |
| 3 | AhrefsBot/7.0 | http://ahrefs.com/robot/ | High | 2025-11-10 03:34:39 | |
| 4 | AliyunSecBot/Aliyun | https://service.alibaba.com | Lower | 2025-11-02 00:24:01 | |
| 5 | Amazonbot/0.1 | https://developer.amazon.com/support/amazonbot | High | 2025-11-10 03:18:13 | |
| 6 | Applebot/0.1 | http://www.apple.com/go/applebot | Medium | 2025-11-10 01:38:12 | |
| 7 | Archive.org_bot | http://archive.org/details/archive.org_bot | Lower | 2025-04-18 01:35:07 | |
| 8 | AwarioBot/1.0 | https://awario.com/bots.html | High | 2025-10-03 08:35:18 | |
| 9 | BLEXBot/1.0 | http://webmeup-crawler.com/ | Lower | 2025-04-18 05:05:08 | |
| 10 | Babbar.tech/crawler | https://babbar.tech/crawler | High | 2025-11-09 23:41:42 | |
| 11 | BacklinksExtendedBot | - | Lower | 2025-04-18 03:55:07 | |
| 12 | Baiduspider/2.0 | http://www.baidu.com/search/spider.html | Lower | 2025-11-06 17:56:20 | |
| 13 | Bingbot/2.0 | http://www.bing.com/bingbot.htm | High | 2025-11-10 03:34:44 | |
| 14 | BitSightBot/1.0 | - | Lower | 2025-10-20 00:15:31 | |
| 15 | Bot/1.1 | http://2ip.io | Lower | 2025-09-23 09:37:03 | |
| 16 | BotH6up4 | - | Lower | 2025-10-09 01:00:06 | |
| 17 | BrightEdge-Crawler/1.0 | https://brightedge.com | Lower | 2025-04-18 02:00:08 | |
| 18 | Bytespider | https://bytedance.com | Medium | 2025-11-10 03:04:36 | |
| 19 | CCBot/2.0 | https://commoncrawl.org/faq/ | Medium | 2025-11-08 04:22:10 | |
| 20 | ChatGPT-User/1.0 | https://openai.com/bot | High | 2025-11-10 01:26:23 |
AdsBot-Google-Mobile
Host:
http://www.google.com/mobile/adsbot.html
Rating:
Lower
Updated(UTC):
2025-05-22 04:00:09
AhrefsBot/7.0
Rating:
High
Updated(UTC):
2025-11-10 03:34:39
Amazonbot/0.1
Host:
https://developer.amazon.com/support/amazonbot
Rating:
High
Updated(UTC):
2025-11-10 03:18:13
Archive.org_bot
Host:
http://archive.org/details/archive.org_bot
Rating:
Lower
Updated(UTC):
2025-04-18 01:35:07
Baiduspider/2.0
Host:
http://www.baidu.com/search/spider.html
Rating:
Lower
Updated(UTC):
2025-11-06 17:56:20
BrightEdge-Crawler/1.0
Rating:
Lower
Updated(UTC):
2025-04-18 02:00:08
Bytespider
Rating:
Medium
Updated(UTC):
2025-11-10 03:04:36
ChatGPT-User/1.0
Rating:
High
Updated(UTC):
2025-11-10 01:26:23