Database

There are currently 317 real crawlers registered in the database.

 

1 - 20 of 209 results
Nr. Name Host Rating Updated Info
1 2IP-Bot/1.1 http://2ip.io Lower 2025-12-01 20:53:32
2 AcademicResearchBot/1.0 - Lower 2025-11-26 02:00:13
3 AdsBot-Google http://www.google.com/adsbot.html Lower 2026-01-07 22:14:21
4 AdsBot-Google-Mobile http://www.google.com/mobile/adsbot.html Lower 2025-05-22 04:00:09
5 AhrefsBot/7.0 https://ahrefs.com/robot/ High 2026-01-19 22:55:19
6 AliyunSecBot/Aliyun https://service.alibaba.com Lower 2026-01-19 21:43:15
7 Amazonbot/0.1 https://developer.amazon.com/support/amazonbot High 2026-01-19 19:43:37
8 Applebot/0.1 http://www.apple.com/go/applebot Medium 2026-01-19 22:39:57
9 Archive.org_bot http://archive.org/details/archive.org_bot Lower 2025-04-18 01:35:07
10 AwarioBot/1.0 https://awario.com/bots.html High 2025-11-21 11:15:49
11 BLEXBot/1.0 http://webmeup-crawler.com/ Lower 2025-04-18 05:05:08
12 BacklinksExtendedBot - Lower 2025-04-18 03:55:07
13 Baiduspider/2.0 http://www.baidu.com/search/spider.html Lower 2026-01-19 22:44:58
14 Barkrowler/0.9 https://babbar.tech/crawler High 2026-01-19 22:55:49
15 Bingbot/2.0 http://www.bing.com/bingbot.htm High 2026-01-19 22:54:15
16 BitSightBot/1.0 https://www.bitsight.com/ Lower 2026-01-07 17:36:20
17 BrightEdge-Crawler/1.0 https://brightedge.com Lower 2025-04-18 02:00:08
18 Bytespider https://bytedance.com Medium 2026-01-19 22:32:23
19 CCBot/2.0 https://commoncrawl.org/faq/ Medium 2026-01-18 08:43:19
20 CareerScraper/1.0 - Lower 2026-01-16 12:34:03

2IP-Bot/1.1

Host:
http://2ip.io

Rating:
Lower

Updated(UTC):
2025-12-01 20:53:32

More information

AcademicResearchBot/1.0

Host:
-

Rating:
Lower

Updated(UTC):
2025-11-26 02:00:13

More information

AdsBot-Google

Host:
http://www.google.com/adsbot.html

Rating:
Lower

Updated(UTC):
2026-01-07 22:14:21

More information

AdsBot-Google-Mobile

Host:
http://www.google.com/mobile/adsbot.html

Rating:
Lower

Updated(UTC):
2025-05-22 04:00:09

More information

AhrefsBot/7.0

Host:
https://ahrefs.com/robot/

Rating:
High

Updated(UTC):
2026-01-19 22:55:19

More information

AliyunSecBot/Aliyun

Host:
https://service.alibaba.com

Rating:
Lower

Updated(UTC):
2026-01-19 21:43:15

More information

Amazonbot/0.1

Host:
https://developer.amazon.com/support/amazonbot

Rating:
High

Updated(UTC):
2026-01-19 19:43:37

More information

Applebot/0.1

Host:
http://www.apple.com/go/applebot

Rating:
Medium

Updated(UTC):
2026-01-19 22:39:57

More information

Archive.org_bot

Host:
http://archive.org/details/archive.org_bot

Rating:
Lower

Updated(UTC):
2025-04-18 01:35:07

More information

AwarioBot/1.0

Host:
https://awario.com/bots.html

Rating:
High

Updated(UTC):
2025-11-21 11:15:49

More information

BLEXBot/1.0

Host:
http://webmeup-crawler.com/

Rating:
Lower

Updated(UTC):
2025-04-18 05:05:08

More information

BacklinksExtendedBot

Host:
-

Rating:
Lower

Updated(UTC):
2025-04-18 03:55:07

More information

Baiduspider/2.0

Host:
http://www.baidu.com/search/spider.html

Rating:
Lower

Updated(UTC):
2026-01-19 22:44:58

More information

Barkrowler/0.9

Host:
https://babbar.tech/crawler

Rating:
High

Updated(UTC):
2026-01-19 22:55:49

More information

Bingbot/2.0

Host:
http://www.bing.com/bingbot.htm

Rating:
High

Updated(UTC):
2026-01-19 22:54:15

More information

BitSightBot/1.0

Host:
https://www.bitsight.com/

Rating:
Lower

Updated(UTC):
2026-01-07 17:36:20

More information

BrightEdge-Crawler/1.0

Host:
https://brightedge.com

Rating:
Lower

Updated(UTC):
2025-04-18 02:00:08

More information

Bytespider

Host:
https://bytedance.com

Rating:
Medium

Updated(UTC):
2026-01-19 22:32:23

More information

CCBot/2.0

Host:
https://commoncrawl.org/faq/

Rating:
Medium

Updated(UTC):
2026-01-18 08:43:19

More information

CareerScraper/1.0

Host:
-

Rating:
Lower

Updated(UTC):
2026-01-16 12:34:03

More information

<< 1 2 3 >>

We use cookies to give you the best possible user experience on our website. By clicking on “Accept” you agree to the use of cookies.

More information Accept