Crawlers database

API - Documentation

The API is very simple!
Simply send a GET request to https://crawlers-info.de/get-crawlers-data?format=json and receive a response in JSON format. The entire database is output in JSON format. If you need an XML format, simply change the GET parameter to format=xml. The entire database is output in XML format.

You can also enter the optional parameter GET rating=medium, then a lower, medium or high rating will be selected accordingly.

cURL:

curl -G -d "format=json" -d "rating=all" https://crawlers-info.de/get-crawlers-data

I recommend saving the received data in a file. Currently, API queries are not throttled, but this may change in the future. Multiple queries per day are unnecessary as database updates are performed once a day. The update time (Europe/Berlin, UTC +2) is between 1:00 and 3:00 a.m. daily.

The platform collects, updates and provides information around the clock completely independently and without human intervention. If you find any incorrect information, please contact us using the contact form.

You are welcome to use the data for your own purposes. However, I would be very grateful if you would support this project in its maintenance and further development. Further information on donations can be found here.