How to crawl a single folder?
Berislav avatar
Written by Berislav
Updated over a week ago

It is possible to crawl a single subfolder with RYTE. If you wish to crawl a single folder on your domain is to open the Project Settings in the top right of your RYTE Account.

There, navigate to "Advanced analysis" > "What to analyze" > "Analyze subfolder". Here you can add your folder. Keep in mind that your folder must be relative to your root directory!
โ€‹

An example would look like this: if we were to crawl our folder "wiki", the URL would look like this: https://en.ryte.com/wiki/. To crawl just the contents located in the wiki folder, we enter /wiki/ in the subfolder field. This is the instruction for the crawler to operate only in the folder /wiki/ and subfolders of /wiki/.

IMPORTANT NOTE: depending on the URL structure of your subdirectory, it's possible that the subdirectory URL in question doesn't contain a trailing slash and may be entered without one in the setting. Therefore, it's important to follow the URL of the subdirectory and the handling of the trailing slash. Otherwise, the crawl may fail if the subdirectory is applied incorrectly.
โ€‹
If you activate the subfolder setting, it is absolutely necessary to edit the homepage URL of the project. Click on "Homepage URL" by staying in the tab "What to analyze".

It is important that the homepage URL contains the directory of the subfolder. If not - the crawl won't be successful. In this example, the homepage URL would be adjusted from https://en.ryte.com/ to https://en.ryte.com/wiki/.


Did this answer your question?