Database
There are currently 1670 real crawlers registered in the database.
| Nr. | Name | Host | Rating | Updated | Info |
|---|---|---|---|---|---|
| 1321 | SiteCheckerBotCrawler/1.0 | http://sitechecker.pro | Lower | 2025-05-24 01:00:06 | |
| 1322 | SiteGuardian/1.4 | https://siteguardian.io/bot | Lower | 2026-06-20 16:35:13 | |
| 1323 | SitesOverPagesBot/1.0 | https://sitesoverpages.com/bot | Lower | 2025-09-11 20:24:24 | |
| 1324 | Slackbot-LinkExpanding | https://api.slack.com/robots | Lower | 2025-11-01 21:34:52 | |
| 1325 | SleepBot/1.0 | http://sleepbot.com/ | High | 2026-08-11 22:05:09 | |
| 1326 | Smart-seo-tools.sbs/bot | https://smart-seo-tools.sbs/bot | Lower | 2026-05-25 13:00:05 | |
| 1327 | SocialMediaScrapperv2/1.0 | https://localhost/social-scraper | Lower | 2026-04-11 08:59:47 | |
| 1328 | Spider/1.0 - No data | - | Lower | 2026-02-21 06:48:51 | |
| 1329 | Spider/20.2 | - | Lower | 2025-04-20 13:00:08 | |
| 1330 | Spider/8.1 | - | Lower | 2025-09-18 04:00:09 | |
| 1331 | Spider_Bot/3.0 | - | Lower | 2025-04-18 07:20:06 | |
| 1332 | Spider_Bot/3.1 | - | Lower | 2025-12-31 00:59:31 | |
| 1333 | Sqalix-cc-scraper-round1/1.0 | - | Lower | 2026-03-23 14:00:05 | |
| 1334 | Star-finder | https://star-finder.de | Lower | 2025-10-29 18:42:23 | |
| 1335 | Statistik-Hessen-Scraping | https://statistik.hessen.de/ua | Lower | 2026-01-06 06:01:35 | |
| 1336 | StepstoneCrawlBot | https://www.thestepstonegroup.com/crawler/ | High | 2026-08-12 13:59:38 | |
| 1337 | StoradarBot/1.0 | https://storadar.com | Lower | 2026-08-06 08:58:54 | |
| 1338 | Storebot-Google/1.0 | - | Lower | 2026-03-26 02:10:49 | |
| 1339 | StorefrontDiscoveryBot/0.1 | https://predict-ability.com | Lower | 2026-07-31 11:03:36 | |
| 1340 | Stripebot/1.0 | https://docs.stripe.com/stripebot-crawler | Lower | 2026-03-31 23:03:20 |
SiteCheckerBotCrawler/1.0
Rating:
Lower
Updated(UTC):
2025-05-24 01:00:06
SitesOverPagesBot/1.0
Host:
https://sitesoverpages.com/bot
Rating:
Lower
Updated(UTC):
2025-09-11 20:24:24
Slackbot-LinkExpanding
Host:
https://api.slack.com/robots
Rating:
Lower
Updated(UTC):
2025-11-01 21:34:52
Smart-seo-tools.sbs/bot
Host:
https://smart-seo-tools.sbs/bot
Rating:
Lower
Updated(UTC):
2026-05-25 13:00:05
SocialMediaScrapperv2/1.0
Host:
https://localhost/social-scraper
Rating:
Lower
Updated(UTC):
2026-04-11 08:59:47
Star-finder
Rating:
Lower
Updated(UTC):
2025-10-29 18:42:23
Statistik-Hessen-Scraping
Host:
https://statistik.hessen.de/ua
Rating:
Lower
Updated(UTC):
2026-01-06 06:01:35
StepstoneCrawlBot
Host:
https://www.thestepstonegroup.com/crawler/
Rating:
High
Updated(UTC):
2026-08-12 13:59:38
StorefrontDiscoveryBot/0.1
Host:
https://predict-ability.com
Rating:
Lower
Updated(UTC):
2026-07-31 11:03:36
Stripebot/1.0
Host:
https://docs.stripe.com/stripebot-crawler
Rating:
Lower
Updated(UTC):
2026-03-31 23:03:20