All Collections
Project Settings
Advanced settings
Where can I change the user-agent?
Where can I change the user-agent?
Berislav avatar
Written by Berislav
Updated over a week ago

In case the crawl did not get any data or seems to be crawling for a long time without finding any URLs, a change of the useragent or crawler country can sometimes help.

To check whether our crawler is able to crawl your domain and find any URLs in the current crawl, have a look at the crawl history: (Project Settings -> Previous analyses)

If 0 URLs are found after a decent amount of time into the crawl you can cancel the crawl and retry with a different User-agent (Advanced analysis > How to analyze > Analysis user-agent):

The option user-agent will determine the name of the crawler. You can enter RyteBot or GoogleBot as user-agent, while the RyteBot is the default user-agent. Or if you want to make sure that you whitelist or track our service only, you can give the crawler an individual name (e.g:crawler123abc)

This option is only available with the Basic Suite or higher!

Did this answer your question?