How to find redirect chains using Screaming Frog

If a site has existed for a couple of years, there’s a good chance that you will find redirect chains on it.

What is a redirect chain?

In simple terms, a redirect chain is formed when URL A redirects to URL B and URL B may further redirect to URL C. This could continue for N number of times. The number of intermediate redirects is commonly called redirect hops.

The processes described below assume you have some basic familiarity with Screaming Frog and how to set up a crawl on it.

How to find redirect chains using Screaming Frog in Spider or List Mode

1. Go to Configuration > Spider > Advanced. Check Always Follow Redirects to true. Set up the rest of the settings as you feel necessary. For details check out Screaming Frog’s official how to guide.
2. Set Max Redirects to Follow number to what you prefer. 5 is default and fine. Run the crawl after all settings are configured.
3. Go to Configuration > Spider > Crawl > Crawl Behavior and set Check Links Outside of Start Folder to true OR set Crawl Outside of Start Folder to true.
   
   Checking either of these 2 settings works but the setting “Crawl outside the start folder” is the one.
   
   Run the crawl after all settings are configured.
4. Go to Reports > Redirects > Redirect Chains and export your file in the desired format.

Why this setting is important when finding redirect chains with Screaming Frog
Imagine the two cases of redirect chains below:

• http://seotesting.cueblocks.com/blog/how-to-be-awesome → http://seotesting.cueblocks.com/blog/how-to-become-awesome → http://seotesting.cueblocks.com/blog/you-are-awesome →
  http://seotesting.cueblocks.com/index.html
  
  
• http://seotesting.cueblocks.com/blog/how-to-be-amazing → http://seotesting.cueblocks.com/blog/how-to-become-amazing → http://seotesting.cueblocks.com/resources/you-are-amazing →
  http://seotesting.cueblocks.com/index.html

In both cases, the redirect chain goes through 3 hops to reach the final destination. However, if you run it in screaming frog and do not set Crawl Outside of Start Folder to true, then Screaming Frog will read the first case as 3 redirects and the second one as 2 redirects.

Case #1 Result – Without enabling “Check Outside of Start Folder”

Case #2 Result – After enabling “Check Outside of Start Folder”

Reason – The folder /resources/ is outside the start folder i.e. /blog/. By setting the Crawl Outside of Start Folder true you are telling Screaming frog to follow the redirects no matter what.

It is incredible how a simple setting can create an almost undetected issue in your analysis.

Screaming frog is an incredible tool. However, without a good understanding of all the features, the tool has to offer, your crawl reports might be incomplete or inaccurate.

Tweet

Rajiv Singha

I am part of the CueBlocks Organic Search(SEO) team. I love playing football and reading about Technical SEO.

MORE POSTS