By Berislav and 1 other
Project Settings Manual
How can I change the sitemap?
Using the blacklist and whitelist (ignore or include URLs)
How can I just crawl a Subdomain?
How to crawl a single folder?
How to change the Homepage URL for my project?
Where can I change the user-agent?