Stop Pulling My (Redirect) Chain! Find & Fix Them, Here’s How

Undoing Redirect Chains = Instant ROI

Want something you can do TODAY that will have an immediate effect on your SEO? Recently Screaming Frog added a report called “redirect chains” to their export options. And if you get a little creative there’s tons of value you can get from this one report. This post is for you if;

• you’re working with a large site
• you’re working with an old site
• you’re working with … you guessed it … an old AND large site.
• your site already has a lot of pages with inbound links
• maybe you’ve purchased or acquired an old site and are fixing it up or redirecting it to a new domain

The only situation I can think of in which this report won’t generate much value, is if you just have a new or small or less-linked to site (or all three factors).

Problems With Redirect Chains

Like the junk drawer of the internet, 301 redirects get thrown somewhere, an .htaccess file maybe, and forgotten about. Yet just because you stuffed those old receipts, fancy pens and unwanted Holiday gifts into that junk drawer … doesn’t mean they’re not still in there waiting to be fixed. Over time, redirects can add up to several redirect “chains”.

Issues caused by those 301 chains include;

• Slow PageSpeed – for users or web crawlers to pass through those 301s, for every one you have, that page takes longer to load or the crawler has to work harder.We all know that site speed is a small ranking factor. And speeding up your site can have numerous benefits. We also know that Google will drop your mobile ranking if your mobile site does not load quickly.
• Poor User Experience – what’s that statistic we hear all the time? Users will bounce if your page takes longer than 3 seconds to load? Improving UX through speeding up your site has been researched and talked about many times. It’s benefits are well documented.
• Lost Link Value? – I believe so. Many interpretations and questions surrounded this video by Matt Cutts “What percentage of PR is lost through a 301 redirect?” – I even recently wrote my own interpretation of Matt’s video here on Google Plus. Bottom line? Chains of 301’s dilute PageRank. PageRank isn’t everything, but if you can preserve it … while at the same time improving SiteSpeed and UX … why NOT?
• Conversion Rates Can Suffer – check out this post on KissMetrics demonstrating how a slow site can even hurt your bottom line.

Part One: Finding Redirect Chains, 4 Methods

The hardest part here can be just coming up with ways to find these crazy buggers. There’s basically five methods for finding redirect chains;

1. Simply crawl the site and export redirect chains report.
2. Export old URLs from Analytics and crawl the URLs in list mode.
3. Export top pages from Open Site Explorer and crawl the URLs in list mode.
4. Crawl the site s archived on archive.org and crawl the URLs in list mode.
5. If you’re lucky enough to have direct access to your list of redirects, you could use that. (We won’t cover this method though, as this won’t always be the case.)
6. EDIT – BONUS: If you are doing a site migration you should always crawl the old site, save the list of old URLs and then follow this post on the Screaming Frog Blog. It follows the same approach, except you are grabbing the old list of URLs yourself before updating the site.

Method 1 – Crawl The Site & Export Redirect Chain Report

This is the easiest way, but also the method to turn up the least amount of redirect chains. Still worth doing especially if you have a large site with a lot of internal links.

Before you crawl the site, check off “always follow redirects”;

Then you just crawl the site as normal and export the “redirect chains” report when finished;

We’ll get to what to do with this report later in the post. For now … let’s move on to method two of finding redirect chains;

Method 2 – Find Old URLs In Analytics & Crawl In List Mode

As noted, simply crawling the existing site isn’t going to turn up everything. If your site is old, you like have redirects built up from years gone by. Let’s find some in Analytics.

First, go back in time as much as you can and for as long of a time period as you want;

Go to your landing pages report;

You can view as many URLs as you want. You could export everything, or maybe the top 500 landing pages. These will be the most linked to anyway, so just play with the amount of URLs is best for the site you’re working on.

Then export;

When you open the spreadsheet you’ll need to add the full absolute URL to the beginning of each page;

Don’t forget to “paste as values” if you want to preserve this list in Excel;

Now paste into a text editor to save as a .txt file for import in Screaming Frog;

Fire up Screaming Frog, Switch to list mode and open the .txt file;

Before hitting crawl, make sure your search depth is set to zero (otherwise it will just keep crawling the site, but we only want to look at these exact URLs);

If you see Screaming Frog crawling more URLs than there were in your list, that’s good actually, it means it is finding some;

Go refresh your cup of coffee. By the time you’re back, you should be ready to export the same “redirect chain” report we did earlier in method one;

Except THIS report will have all chains coming from OLD URLs maybe you didn’t know even existed anymore.

Method 3 – Top Pages From Open Site Explorer & List Mode

You may find similar URLs in each of these methods by they way, but all methods have their pros and cons, so still worth covering.

Anyhow, let’s move on to Open Site Explorer. Here are the top linked to pages on www.amazon.com;

I’m just using Amazon as an example – but what we’re looking for, is just evidence of any linked to pages that 301, 302, 404 – or anything besides a 200 really. If there’s one 301 etc it could lead to a chain.

So go ahead and export the top pages report to CSV. Fill up that coffee again (or maybe a few cups, these OSE reports can take a few minutes on large sites).

Alright – let’s see what we have. Sort your top pages by HTTP status to bring the bad ones to the top;

This revealed about 3,000 URLs. We’re not going to check all of those, just a small sample of Amazon’s most linked to URLs returning 301 codes;

After running in list mode through Screaming Frog – what do we find?

I was a little shocked to see out of only about 200 URLs, a bunch of redirect chains, including a chain of SIX. But this can’t have many inbound links pointing to it right??

WRONG. How about 921 Linking root domains passing through a chain of six redirects?

That’s what we call in the SEO world … faible consommation de fruits suspendus!! (look it up).

Method 4 – Archive.org & Crawl In List Mode

Last method, and one of my favorites. We’re going to crawl archive.org for old URLs on your site and then run those through Screaming Frog in list mode.

This method is killer if you don’t have access to anything for the site – no analytics, no internal access, and maybe it’s even an old site you’re acquired.

Find the homepage of your site on archive.org – I’m going to use apple.com for this last example;

WOW – CD-RW drives! Those were the days.

Anyhow, that whole archive.org URL is what we want. Copy that to your clipboard. Paste that URL into Screaming Frog, and check the box “ignore robots.txt”;

Export your crawl into CSV;

Open the CSV and move the URLs to your text editor;

Do whatever you need to do to clean up the URLs;

To my surprise, out of only ONE date from archive.org of only 37 URLs – I found a few redirect chains;

Click to Enlarge

I mean … this is APPLE. I thought they’d be on this stuff. The links with redirect chains even have a decent number of back links pointing at them;

And;

I mean, I know this is Apple – and how many links do they really need? Well you better bet I’d be cleaning up stuff like this as best practice. There’s bound to be a cumulative effect.

As noted, you could also get a list of your redirects internally, but we’re not going to cover that in this post.

Part Two: Prepping The Screaming Frog Export

This step follows after ANY and ALL of the above methods of finding the redirect chains. We want to clean up the Screaming Frog export to pass along to a developer or to fix ourselves.

I like to make the lives of developers as EASY as possible. You’re more likely to get the job done, and everyone wins.

Upload the Screaming Frog redirect chain export to Google Docs;

SORT the list of URLs by “number of redirects”;

You can delete rows with only one redirect, as this is technically not a “chain”;

Next is up to you specifically, but I like to delete more columns that are not needed, as well as add some extra heading for clarity for when the developer uses the sheet;

Click To Enlarge

Now it’s ready to pass to the dev or whoever to fix!

Part Three: GRAND Finale. ACTUALLY Fixing Them

If that felt like a lot of steps just to lead up to fixing what we set out to do – well, it was! But we don’t get reward without some effort.

1. Determine IF You Should Undo Them At All

Most likely you should, but as Cyrus Shepard wrote about a few months back, 301’s can both help and hurt your site.

I’m not going to cover everything in his post here, but essentially, 301’s that are relevant and topically similar in content are great. 301’s that have no relevance don’t do much of anything at all.

2. If All Looks Good – Undo Them!

Here’s what your chain looks like;

That’s the shortest chain you can have, of only two redirects. Remember in one Amazon example we found a chain of six.

You’re essentially going to undo them like this;

Yes, This Is Technical – But It’s “Bigger” Than Just PageRank & Site Speed

Source: http://www.pinterest.com/pin/185984659585168432/

You’ve heard of a content audit right? And you’re heard of … oh … that thing lately called “Content Strategy“? What’s that have to do with 301 redirect chains?

Content audits and undoing redirect chains ONLY exist because of the internet’s age. Think about it. What would we have content audited in 2002? There’s a whole lot more content in 2013 than 2002.

And there’s wayyy more redirects on the web now than just a few years ago.

Yes folks, the future is coming. A day when a content audit – or “redirect chain audit” are more common realities and services than some new thing only a few people are catching onto.

The web is a physical place as much as Rome or Athens or New York City. It has a history, old roads, old infrastructure – and over time these things will need attention. Who’s going to do that first? You? Or your competitors?