Why was my project not crawled completely?

When your project was not crawled completely please have a look into the Project Settings -> Advanced Crawl to find out what's missing from the crawl:

CC.png 

In this example we see that some URLs were ignored, this can be due to unintended crawl settings.

In case you activated Obey robots.txt all "disallowed" URLs will be ignored, deactivated subdomains will lead to ignored URLs to your subdomains and so on.  Please also have a look at the other settings such as Blacklist/Whitelist, crawl subfolder and analyze images.

 

Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.
Powered by Zendesk