Ignore / Include URLs

1. Blacklist/Whitelist

include-exlude-white-black-list2.png

You can exclude URLs from the analysis by adding blacklist rules. In this example we want to exclude our Magazine and Wiki, we can realize that with two lines of regex:

regex:\/wiki\/
regex:\/magazine\/

Please note that this will also affect domain.com/subfolder/wiki/, you may need to adjust it in depth (e.g. regex:https:\/\/en.ryte.com\/wiki\/)

You can apply any rules by adding regular expressions such as:

Certain URL:

regex:https:\/\/en.ryte.com\/magazine\/onpage-becomes-ryte

Certain string within a URL:

regex:urlpart

Filetype:

regex:^.*\.(jpg|JPG|gif|GIF|doc|DOC|pdf|PDF|js)$


To check if your line has the right syntax you can run it through a regex validator
(enter without "regex:")

Handy metacharacters:

^   asserts position at the start of the string.

$   asserts position at the end of the string.

*   Quantifier — Matches between zero and unlimited times, as many times as possible, giving back as needed.

.   matches any character.

 

The whitelist works the opposite way ("analyze only"). If you want to analyze a specific part of your domain or narrow it down by certain criteria you can realize that with the whitelist.

 

This is how you can test your whitelist/blacklist rules.

 

2. Subfolder

cs1.png

If you want to analyze a specific subfolder only please use the "Analyze subfolder"-Option instead. (e.g. "/wiki/)


Have more questions? Submit a request

0 Comments

Please sign in to leave a comment.
Powered by Zendesk