Undoing Redirect Chains = Instant ROI

Want something you can do TODAY that will have an immediate effect on your SEO? Recently Screaming Frog added a report called “redirect chains” to their export options. And if you get a little creative there’s tons of value you can get from this one report. This post is for you if;

  • you’re working with a large site
  • you’re working with an old site
  • you’re working with … you guessed it … an old AND large site.
  • your site already has a lot of pages with inbound links
  • maybe you’ve purchased or acquired an old site and are fixing it up or redirecting it to a new domain

The only situation I can think of in which this report won’t generate much value, is if you just have a new or small or less-linked to site (or all three factors).

Problems With Redirect Chains

Like the junk drawer of the internet, 301 redirects get thrown somewhere, an .htaccess file maybe, and forgotten about. Yet just because you stuffed those old receipts, fancy pens and unwanted Holiday gifts into that junk drawer … doesn’t mean they’re not still in there waiting to be fixed. Over time, redirects can add up to several redirect “chains”.

Issues caused by those 301 chains include;

  • Slow PageSpeed – for users or web crawlers to pass through those 301s, for every one you have, that page takes longer to load or the crawler has to work harder.We all know that site speed is a small ranking factor. And speeding up your site can have numerous benefits. We also know that Google will drop your mobile ranking if your mobile site does not load quickly.
  • Poor User Experience – what’s that statistic we hear all the time? Users will bounce if your page takes longer than 3 seconds to load? Improving UX through speeding up your site has been researched and talked about many times. It’s benefits are well documented.
  • Lost Link Value? – I believe so. Many interpretations and questions surrounded this video by Matt Cutts “What percentage of PR is lost through a 301 redirect?” – I even recently wrote my own interpretation of Matt’s video here on Google Plus. Bottom line? Chains of 301′s dilute PageRank. PageRank isn’t everything, but if you can preserve it … while at the same time improving SiteSpeed and UX … why NOT?
  • Conversion Rates Can Suffer – check out this post on KissMetrics demonstrating how a slow site can even hurt your bottom line.

Part One: Finding Redirect Chains, 4 Methods

The hardest part here can be just coming up with ways to find these crazy buggers. There’s basically five methods for finding redirect chains;

  1. Simply crawl the site and export redirect chains report.
  2. Export old URLs from Analytics and crawl the URLs in list mode.
  3. Export top pages from Open Site Explorer and crawl the URLs in list mode.
  4. Crawl the site s archived on archive.org and crawl the URLs in list mode.
  5. If you’re lucky enough to have direct access to your list of redirects, you could use that. (We won’t cover this method though, as this won’t always be the case.)
  6. EDIT – BONUS: If you are doing a site migration you should always crawl the old site, save the list of old URLs and then follow this post on the Screaming Frog Blog. It follows the same approach, except you are grabbing the old list of URLs yourself before updating the site.

Method 1 – Crawl The Site & Export Redirect Chain Report

This is the easiest way, but also the method to turn up the least amount of redirect chains. Still worth doing especially if you have a large site with a lot of internal links.

Before you crawl the site, check off “always follow redirects”;

check off always follow redirects

Then you just crawl the site as normal and export the “redirect chains” report when finished;

crawl and export redirect chains report

We’ll get to what to do with this report later in the post. For now … let’s move on to method two of finding redirect chains;

Method 2 – Find Old URLs In Analytics & Crawl In List Mode

As noted, simply crawling the existing site isn’t going to turn up everything. If your site is old, you like have redirects built up from years gone by. Let’s find some in Analytics.

First, go back in time as much as you can and for as long of a time period as you want;

adjust dates in analytics

Go to your landing pages report;

view landing pages as many as you want

You can view as many URLs as you want. You could export everything, or maybe the top 500 landing pages. These will be the most linked to anyway, so just play with the amount of URLs is best for the site you’re working on.

Then export;

export landing pages

When you open the spreadsheet you’ll need to add the full absolute URL to the beginning of each page;

concatenate full urls in excel

Don’t forget to “paste as values” if you want to preserve this list in Excel;

paste special to preserve absolute urls

Now paste into a text editor to save as a .txt file for import in Screaming Frog;

paste into text editor

Fire up Screaming Frog, Switch to list mode and open the .txt file;

run screaming frog in list mode

Before hitting crawl, make sure your search depth is set to zero (otherwise it will just keep crawling the site, but we only want to look at these exact URLs);

set search depth to zero

If you see Screaming Frog crawling more URLs than there were in your list, that’s good actually, it means it is finding some;

number of urls crawled is higher

Go refresh your cup of coffee. By the time you’re back, you should be ready to export the same “redirect chain” report we did earlier in method one;

export redirect chain report

Except THIS report will have all chains coming from OLD URLs maybe you didn’t know even existed anymore.

Method 3 – Top Pages From Open Site Explorer & List Mode

You may find similar URLs in each of these methods by they way, but all methods have their pros and cons, so still worth covering.

Anyhow, let’s move on to Open Site Explorer. Here are the top linked to pages on www.amazon.com;

top linked to pages on amazon.com

I’m just using Amazon as an example – but what we’re looking for, is just evidence of any linked to pages that 301, 302, 404 – or anything besides a 200 really. If there’s one 301 etc it could lead to a chain.

So go ahead and export the top pages report to CSV. Fill up that coffee again (or maybe a few cups, these OSE reports can take a few minutes on large sites).

Alright – let’s see what we have. Sort your top pages by HTTP status to bring the bad ones to the top;

sort top pages by http status descending

This revealed about 3,000 URLs. We’re not going to check all of those, just a small sample of Amazon’s most linked to URLs returning 301 codes;

we'll run a sample of top pages

After running in list mode through Screaming Frog – what do we find?

amazon bad links with redirect chains

I was a little shocked to see out of only about 200 URLs, a bunch of redirect chains, including a chain of SIX. But this can’t have many inbound links pointing to it right??

WRONG. How about 921 Linking root domains passing through a chain of six redirects?

url on amazon with 921 lrds

That’s what we call in the SEO world … faible consommation de fruits suspendus!! (look it up).

Method 4 – Archive.org & Crawl In List Mode

Last method, and one of my favorites. We’re going to crawl archive.org for old URLs on your site and then run those through Screaming Frog in list mode.

This method is killer if you don’t have access to anything for the site – no analytics, no internal access, and maybe it’s even an old site you’re acquired.

Find the homepage of your site on archive.org – I’m going to use apple.com for this last example;

apple.com archive.org screenshot

WOW – CD-RW drives! Those were the days.

Anyhow, that whole archive.org URL is what we want. Copy that to your clipboard. Paste that URL into Screaming Frog, and check the box “ignore robots.txt”;

crawling archive.org

Export your crawl into CSV;

export crawl of old URLs

Open the CSV and move the URLs to your text editor;

move urls to text editor

Do whatever you need to do to clean up the URLs;

clean up list of old urls

To my surprise, out of only ONE date from archive.org of only 37 URLs – I found a few redirect chains;

bad apple links!

Click to Enlarge

I mean … this is APPLE. I thought they’d be on this stuff. The links with redirect chains even have a decent number of back links pointing at them;

old url with linking root domains on apple.com

And;

another bad url with links pointing at it on apple.com

I mean, I know this is Apple – and how many links do they really need? Well you better bet I’d be cleaning up stuff like this as best practice. There’s bound to be a cumulative effect.

As noted, you could also get a list of your redirects internally, but we’re not going to cover that in this post.

Part Two: Prepping The Screaming Frog Export

This step follows after ANY and ALL of the above methods of finding the redirect chains. We want to clean up the Screaming Frog export to pass along to a developer or to fix ourselves.

I like to make the lives of developers as EASY as possible. You’re more likely to get the job done, and everyone wins.

Upload the Screaming Frog redirect chain export to Google Docs;

upload report as google spreadsheet

SORT the list of URLs by “number of redirects”;

sort by number of redirects

You can delete rows with only one redirect, as this is technically not a “chain”;

delete rows with only one redirect

Next is up to you specifically, but I like to delete more columns that are not needed, as well as add some extra heading for clarity for when the developer uses the sheet;

finish cleaning up spreadsheet

Click To Enlarge

Now it’s ready to pass to the dev or whoever to fix!

Part Three: GRAND Finale. ACTUALLY Fixing Them

If that felt like a lot of steps just to lead up to fixing what we set out to do – well, it was! But we don’t get reward without some effort.

1. Determine IF You Should Undo Them At All

Most likely you should, but as Cyrus Shepard wrote about a few months back, 301′s can both help and hurt your site.

I’m not going to cover everything in his post here, but essentially, 301′s that are relevant and topically similar in content are great. 301′s that have no relevance don’t do much of anything at all.

2. If All Looks Good – Undo Them!

Here’s what your chain looks like;

chain of two redirects

That’s the shortest chain you can have, of only two redirects. Remember in one Amazon example we found a chain of six.

You’re essentially going to undo them like this;

fixed chain of two redirects

Yes, This Is Technical – But It’s “Bigger” Than Just PageRank & Site Speed

an abandoned building as metaphor for abandoned websites

Source: http://www.pinterest.com/pin/185984659585168432/

You’ve heard of a content audit right? And you’re heard of … oh … that thing lately called “Content Strategy“? What’s that have to do with 301 redirect chains?

Content audits and undoing redirect chains ONLY exist because of the internet’s age. Think about it. What would we have content audited in 2002? There’s a whole lot more content in 2013 than 2002.

And there’s wayyy more redirects on the web now than just a few years ago.

Yes folks, the future is coming. A day when a content audit – or “redirect chain audit” are more common realities and services than some new thing only a few people are catching onto.

The web is a physical place as much as Rome or Athens or New York City. It has a history, old roads, old infrastructure – and over time these things will need attention. Who’s going to do that first? You? Or your competitors?

About Dan Shure

I'm Dan (Google Plus Profile). I've been helping businesses improve their websites since 2007. Improving your bottom line is my number one goal. My obsessive nature and love for SEO as a true craft doesn't hurt either.

14 Comments

  • October 15, 2013 Reply

    Jake

    Wow! I have my work cut out for me!

    Thank you for sharing

    • October 15, 2013 Reply

      Dan Shure

      No problem :)

  • October 18, 2013 Reply

    Yasir

    Hi Dan,

    Great info. I really like the methods you mentioned. I use SF but to find redirected chains, I mainly use IIS SEO toolkit which I think is better than SF at finding redirected chains but it does lack the list mode which is a great feature in SF.

    In addition to your 4 methods, I think the Internal Links section in GWT is also a good place to find links that you can run this redirected chain report for to find potential issues.

    • October 18, 2013 Reply

      Dan Shure

      Ohhh good idea! That makes sense – I should try that :)

  • October 29, 2013 Reply

    BrettASnyder

    Nice job putting this together Dan, I think we as SEOs can have a tendency to look for “the next big thing” to “stay ahead of the curve” to the point of sometimes forgeting the faible consommation de fruits suspendus (I did have to look it up haha) that are a lot more tactical but still generate measurable results. Cheers!

    • October 30, 2013 Reply

      Dan Shure

      I approve of the French. And yeah I totally agree, we’re always looking for the new shiny thing, however devs and so forth can leave a mess behind sometimes and it’s our job to clean it up ;) (No offense to any devs reading this)

  • October 29, 2013 Reply

    Bryan Vu

    Thanks for the tips Dan, and for a well-written post!

    Using OSE I did find a few extra errors vs. just crawling the site. Also depending on when Analytics was implemented, it could pay off to combine page lists from multiple sources and removing duplicates before running a list crawl.

    • October 30, 2013 Reply

      Dan Shure

      Hey you bet man! Do me a favor, keep Ross in line over there :)

  • June 9, 2014 Reply

    Andre

    Hi Dan,

    Cyrus covered the theory. You covered the practice. Good combo. Thanks.

    We have good reason to believe that some mass redirecting when we consolidated 3 sites into one, led to a Google manual action. Will the above approach help us identify external redirects, or only internals?

    If not, what’s the best tool / method for identifying external redirects?

    Andre

    • June 9, 2014 Reply

      Dan Shure

      Hey Andre – can you extrapolate a little on what you mean by “external redirects”? This method I believe will only address redirects within one domain btw.

      • July 21, 2014 Reply

        Andre

        I was referring to redirects that traverse over multiple domains. domainA -> domainB -> domainC.

        Thanks for getting back to me (and sorry for the slow response).

  • June 29, 2014 Reply

    Fahad rehman

    Hey man, well the problem that arises for me is that i want to remove redirect chains from a blogger blog. Any help how that could be done. My blog is hotmobiledeals.blogspot.com

Leave a Comment

Error: Please check your entries!