Undoing Redirect Chains = Instant ROI
Want something you can do TODAY that will have an immediate effect on your SEO? Recently Screaming Frog added a report called “redirect chains” to their export options. And if you get a little creative there’s tons of value you can get from this one report. This post is for you if;
- you’re working with a large site
- you’re working with an old site
- you’re working with … you guessed it … an old AND large site.
- your site already has a lot of pages with inbound links
- maybe you’ve purchased or acquired an old site and are fixing it up or redirecting it to a new domain
The only situation I can think of in which this report won’t generate much value, is if you just have a new or small or less-linked to site (or all three factors).
Problems With Redirect Chains
Like the junk drawer of the internet, 301 redirects get thrown somewhere, an .htaccess file maybe, and forgotten about. Yet just because you stuffed those old receipts, fancy pens and unwanted Holiday gifts into that junk drawer … doesn’t mean they’re not still in there waiting to be fixed. Over time, redirects can add up to several redirect “chains”.
Issues caused by those 301 chains include;
- Slow PageSpeed – for users or web crawlers to pass through those 301s, for every one you have, that page takes longer to load or the crawler has to work harder.We all know that site speed is a small ranking factor. And speeding up your site can have numerous benefits. We also know that Google will drop your mobile ranking if your mobile site does not load quickly.
- Poor User Experience – what’s that statistic we hear all the time? Users will bounce if your page takes longer than 3 seconds to load? Improving UX through speeding up your site has been researched and talked about many times. It’s benefits are well documented.
- Lost Link Value? – I believe so. Many interpretations and questions surrounded this video by Matt Cutts “What percentage of PR is lost through a 301 redirect?” – I even recently wrote my own interpretation of Matt’s video here on Google Plus. Bottom line? Chains of 301’s dilute PageRank. PageRank isn’t everything, but if you can preserve it … while at the same time improving SiteSpeed and UX … why NOT?
- Conversion Rates Can Suffer – check out this post on KissMetrics demonstrating how a slow site can even hurt your bottom line.
Part One: Finding Redirect Chains, 4 Methods
The hardest part here can be just coming up with ways to find these crazy buggers. There’s basically five methods for finding redirect chains;
- Simply crawl the site and export redirect chains report.
- Export old URLs from Analytics and crawl the URLs in list mode.
- Export top pages from Open Site Explorer and crawl the URLs in list mode.
- Crawl the site s archived on archive.org and crawl the URLs in list mode.
- If you’re lucky enough to have direct access to your list of redirects, you could use that. (We won’t cover this method though, as this won’t always be the case.)
- EDIT – BONUS: If you are doing a site migration you should always crawl the old site, save the list of old URLs and then follow this post on the Screaming Frog Blog. It follows the same approach, except you are grabbing the old list of URLs yourself before updating the site.
Method 1 – Crawl The Site & Export Redirect Chain Report
This is the easiest way, but also the method to turn up the least amount of redirect chains. Still worth doing especially if you have a large site with a lot of internal links.
Before you crawl the site, check off “always follow redirects”;
Then you just crawl the site as normal and export the “redirect chains” report when finished;
We’ll get to what to do with this report later in the post. For now … let’s move on to method two of finding redirect chains;
Method 2 – Find Old URLs In Analytics & Crawl In List Mode
As noted, simply crawling the existing site isn’t going to turn up everything. If your site is old, you like have redirects built up from years gone by. Let’s find some in Analytics.
First, go back in time as much as you can and for as long of a time period as you want;
Go to your landing pages report;
You can view as many URLs as you want. You could export everything, or maybe the top 500 landing pages. These will be the most linked to anyway, so just play with the amount of URLs is best for the site you’re working on.
When you open the spreadsheet you’ll need to add the full absolute URL to the beginning of each page;
Don’t forget to “paste as values” if you want to preserve this list in Excel;
Now paste into a text editor to save as a .txt file for import in Screaming Frog;
Fire up Screaming Frog, Switch to list mode and open the .txt file;
Before hitting crawl, make sure your search depth is set to zero (otherwise it will just keep crawling the site, but we only want to look at these exact URLs);
If you see Screaming Frog crawling more URLs than there were in your list, that’s good actually, it means it is finding some;
Go refresh your cup of coffee. By the time you’re back, you should be ready to export the same “redirect chain” report we did earlier in method one;
Except THIS report will have all chains coming from OLD URLs maybe you didn’t know even existed anymore.
Method 3 – Top Pages From Open Site Explorer & List Mode
You may find similar URLs in each of these methods by they way, but all methods have their pros and cons, so still worth covering.
Anyhow, let’s move on to Open Site Explorer. Here are the top linked to pages on www.amazon.com;
I’m just using Amazon as an example – but what we’re looking for, is just evidence of any linked to pages that 301, 302, 404 – or anything besides a 200 really. If there’s one 301 etc it could lead to a chain.
So go ahead and export the top pages report to CSV. Fill up that coffee again (or maybe a few cups, these OSE reports can take a few minutes on large sites).
Alright – let’s see what we have. Sort your top pages by HTTP status to bring the bad ones to the top;
This revealed about 3,000 URLs. We’re not going to check all of those, just a small sample of Amazon’s most linked to URLs returning 301 codes;
After running in list mode through Screaming Frog – what do we find?
I was a little shocked to see out of only about 200 URLs, a bunch of redirect chains, including a chain of SIX. But this can’t have many inbound links pointing to it right??
WRONG. How about 921 Linking root domains passing through a chain of six redirects?
That’s what we call in the SEO world … faible consommation de fruits suspendus!! (look it up).
Method 4 – Archive.org & Crawl In List Mode
Last method, and one of my favorites. We’re going to crawl archive.org for old URLs on your site and then run those through Screaming Frog in list mode.
This method is killer if you don’t have access to anything for the site – no analytics, no internal access, and maybe it’s even an old site you’re acquired.
Find the homepage of your site on archive.org – I’m going to use apple.com for this last example;
WOW – CD-RW drives! Those were the days.
Anyhow, that whole archive.org URL is what we want. Copy that to your clipboard. Paste that URL into Screaming Frog, and check the box “ignore robots.txt”;
Export your crawl into CSV;
Open the CSV and move the URLs to your text editor;
Do whatever you need to do to clean up the URLs;
To my surprise, out of only ONE date from archive.org of only 37 URLs – I found a few redirect chains;
I mean … this is APPLE. I thought they’d be on this stuff. The links with redirect chains even have a decent number of back links pointing at them;
I mean, I know this is Apple – and how many links do they really need? Well you better bet I’d be cleaning up stuff like this as best practice. There’s bound to be a cumulative effect.
As noted, you could also get a list of your redirects internally, but we’re not going to cover that in this post.
Part Two: Prepping The Screaming Frog Export
This step follows after ANY and ALL of the above methods of finding the redirect chains. We want to clean up the Screaming Frog export to pass along to a developer or to fix ourselves.
I like to make the lives of developers as EASY as possible. You’re more likely to get the job done, and everyone wins.
Upload the Screaming Frog redirect chain export to Google Docs;
SORT the list of URLs by “number of redirects”;
You can delete rows with only one redirect, as this is technically not a “chain”;
Next is up to you specifically, but I like to delete more columns that are not needed, as well as add some extra heading for clarity for when the developer uses the sheet;
Now it’s ready to pass to the dev or whoever to fix!
Part Three: GRAND Finale. ACTUALLY Fixing Them
If that felt like a lot of steps just to lead up to fixing what we set out to do – well, it was! But we don’t get reward without some effort.
1. Determine IF You Should Undo Them At All
Most likely you should, but as Cyrus Shepard wrote about a few months back, 301’s can both help and hurt your site.
I’m not going to cover everything in his post here, but essentially, 301’s that are relevant and topically similar in content are great. 301’s that have no relevance don’t do much of anything at all.
2. If All Looks Good – Undo Them!
Here’s what your chain looks like;
That’s the shortest chain you can have, of only two redirects. Remember in one Amazon example we found a chain of six.
You’re essentially going to undo them like this;
Yes, This Is Technical – But It’s “Bigger” Than Just PageRank & Site Speed
You’ve heard of a content audit right? And you’re heard of … oh … that thing lately called “Content Strategy“? What’s that have to do with 301 redirect chains?
Content audits and undoing redirect chains ONLY exist because of the internet’s age. Think about it. What would we have content audited in 2002? There’s a whole lot more content in 2013 than 2002.
And there’s wayyy more redirects on the web now than just a few years ago.
Yes folks, the future is coming. A day when a content audit – or “redirect chain audit” are more common realities and services than some new thing only a few people are catching onto.
The web is a physical place as much as Rome or Athens or New York City. It has a history, old roads, old infrastructure – and over time these things will need attention. Who’s going to do that first? You? Or your competitors?
October 15, 2013
Wow! I have my work cut out for me!
Thank you for sharing
October 15, 2013
No problem 🙂
October 18, 2013
Great info. I really like the methods you mentioned. I use SF but to find redirected chains, I mainly use IIS SEO toolkit which I think is better than SF at finding redirected chains but it does lack the list mode which is a great feature in SF.
In addition to your 4 methods, I think the Internal Links section in GWT is also a good place to find links that you can run this redirected chain report for to find potential issues.
October 18, 2013
Ohhh good idea! That makes sense – I should try that 🙂
October 29, 2013
Nice job putting this together Dan, I think we as SEOs can have a tendency to look for “the next big thing” to “stay ahead of the curve” to the point of sometimes forgeting the faible consommation de fruits suspendus (I did have to look it up haha) that are a lot more tactical but still generate measurable results. Cheers!
October 30, 2013
I approve of the French. And yeah I totally agree, we’re always looking for the new shiny thing, however devs and so forth can leave a mess behind sometimes and it’s our job to clean it up 😉 (No offense to any devs reading this)
October 29, 2013
Thanks for the tips Dan, and for a well-written post!
Using OSE I did find a few extra errors vs. just crawling the site. Also depending on when Analytics was implemented, it could pay off to combine page lists from multiple sources and removing duplicates before running a list crawl.
October 30, 2013
Hey you bet man! Do me a favor, keep Ross in line over there 🙂
June 9, 2014
Cyrus covered the theory. You covered the practice. Good combo. Thanks.
We have good reason to believe that some mass redirecting when we consolidated 3 sites into one, led to a Google manual action. Will the above approach help us identify external redirects, or only internals?
If not, what’s the best tool / method for identifying external redirects?
June 9, 2014
Hey Andre – can you extrapolate a little on what you mean by “external redirects”? This method I believe will only address redirects within one domain btw.
July 21, 2014
I was referring to redirects that traverse over multiple domains. domainA -> domainB -> domainC.
Thanks for getting back to me (and sorry for the slow response).
June 29, 2014
Hey man, well the problem that arises for me is that i want to remove redirect chains from a blogger blog. Any help how that could be done. My blog is hotmobiledeals.blogspot.com
June 29, 2014
Hi – as far as I know the process of finding them is exactly the same, but I am not familiar with how to edit/update redirects in Blogspot. You might find some info here: https://support.google.com/blogger/answer/2472689?hl=en
September 14, 2014
I have a list of redirects, they are all for images that are smaller than the image used on WordPress. It appears WordPress keeps a smaller copy and does a redirect. I don’t know … I have the whole list and pulled it up through GT Metrix. What do I do now that I have the list? When I go to the URL, these photos are not in my media page. They are all images.
September 15, 2014
Thanks Dan! I was looking for a clear and simple solution and your part 3 did exactly that 🙂
October 3, 2014
Very helpful post!!
We have a site that has the original domain 301 to a new domain (corporate branding) and now the http version 301 to the https.
original site >>> new site >>> https version
Do we need to undo this? I also am unable to update the site settings in GWT because of the change of address that was submitted. Our search queries are tanking while actual organic search is skyrocketig. ha.
Any idea would be super appreciated!
October 6, 2014
Wow this is a great tutorial. I love finding things like this. Great job Dan. I am a big fan of the Redirect Chains report in Screaming Frog. I have a follow up question: Can Screaming Frog be used to detect redirect chains related to 302 redirects in addition to 301 redirects?
October 6, 2014
Thanks, and yup! Screaming Frog will find all 3xx redirects (307s too!)
October 9, 2014
Thanks Dan! Can you please help with following scenario:
url A > url B > url C and now we need url C to be https
Would you recommend the following breakout:
url A > https url C
url B > https url C
url C > https url C
The goal is to maintain as much page rank as possible and have the fastest site experience. Any insight would be much appreciated.
October 9, 2014
Hi – yes that’s exactly the method. That eliminates the chains and links everything directly to the final 200 page.
June 30, 2016
Just wrote an article about 301 redirect chains. Happened to use your great article as a reference.
Would love if you took a look. Basically I also answer the question whether redirect chains hurt SEO (yes) and show a couple of different ways people can fix this issue.
You’ll probably agree – this is a topic that isn’t covered very much! But can have huge impacts.
December 2, 2016
Paal Joachim Romdahl
Thank you for the article!
I am checking the following site on pindgom:
I see the redirect chains are:
https://ssl.google-analytics.com/r/__utm.gif?ut … 418330&utmredir=1&utmu=qAAAAAAAAAAAAAAAAAAAAAAE~
https://stats.g.doubleclick.net/r/collect?v=1&a … 9.1480700940&jid=411418330&_v=5.6.7&z=1809838025
https://www.google.com/ads/ga-audiences?v=1&aip … 9.1480700940&jid=411418330&_v=5.6.7&z=1809838025
What is a good and simple way to remove these redirects?
April 4, 2017
Hey thank you for the article! I had a question that has been stumping me for a little while now.
I have a website that used to be www and http a year or two ago. Now it is https and non www. A lot of my older links point to the www and http version of my site. This results in two 301 redirects.
A link on another site to my site points to http://www.mysite.com
The network waterfall shows:
http://www.mysite.com 301 -> http://mysite.com
http://mysite.com 301 -> https://mysite.com
2 part question.
–Do you think that this two 301 redirect hop would affect SEO performance? I can see it did affect page authority through Moz.
–Is there away around this? I.e. to redirect http:// AND http://www directly to https:// with no hops in between.
April 5, 2017
1. Yes, you definitely want to eliminate these chains.
2. Yes that’s the fix – ‘undo’ the redirect chain by having two separate redirects, just as you described 🙂
April 14, 2017
Thanks for all this great info for finding the redirects, but how do you actually fix them? I have all my redirects and I’m just staring at them now. Do you make the changes through WP, or cpanel, or is it something super geeky that I will need to pay someone to do?
April 14, 2017
How are your current redirects being done? It would either be with a plugin or in your .htaccess file (which can be edited via the Yoast SEO plugin). I would create/update redirects with the same method your current ones are being done.
April 25, 2017
I don’t agree, look at http://blog.catchpoint.com/2016/08/19/revisiting-anatomy-http-part/
April 26, 2017
What part do you not agree with?
April 26, 2017
Hi Dan, I’m Dan too! 🙂
How do I stop this attack on my website please? https://thehackerblog.com/tag/open-redirect-chain/ This link shows the attack its not my website. At least I think I’m under attack, not sure. If this is not an attack, how do I remove all these external redirects that I do NOT own?
April 26, 2017
Hmmm… unfortunately I’m not much of an expert when it comes to blocking attacks. Maybe I shouldn’t say that publically, lol. What do you mean by external redirect?
October 8, 2017
Have you had any luck doing this with the Google analytics redirect chains at all? I’d be interested to hear if you have so many people are using pingdom tools and getting that error and have no idea how to fix it or if they even need to.
Leave a Comment