I recently was faced with a decision: which WordPress caching plugin to use on my shared hosting account. The choice was between WP Super Cache, W3 Total Cache and Hyper Cache. This article documents my admittedly limited testing of these plugins and the decision I made as a result.

The Disclaimer

I’m very aware (and you should be to) that:

• I’m not an expert on caching
• My method of testing is far from scientific
• The website used in my test isn’t well optimised in it’s own right.
• Results could be very different for other websites
• Results could be very different if I spent the time to tweak the settings of each plugin
• Future versions of the caching plugins may improve their performance. Please note their version number below and do your own testing if there are newer versions.

Nonetheless, I present this in case it’s useful for someone. At the minimum, I hope it will encourage people to do their own testing in their environment rather than just believe everything they read.

The Requirements

It was important to me to use a WordPress caching solution that worked well with my server environment: A shared hosting server with Hostgator.

Unfortunately, I’m not on a dedicated server at this point, so benefits such as a persistent object cache are not available to me. If I was on a dedicated server, the results of my testing may have been very different – but I’m not and I need a solution that fits my current circumstances.

It’s also worth pointing out that I’m not trying to protect the server from the Digg effect or from high traffic per se. My sites don’t draw that much traffic. I’m just after a caching solution that will speed up the delivery of my web pages.

The Candidates

There are many WordPress caching plugins available, but I can’t test them all. I settled on these three plugins for the following reasons:

WP Super Cache (0.9.9.3)

WP Super Cache is the current market leader, with 12,182 downloads over the last week (at the time of writing). It’s been the standard for several years now, has been widely written about and has a very good reputation. I’ve personally been using WP Super Cache for a year or so now and it’s served me well, so it was a no brainer to consider continuing with this plugin.

WP Super Cache is written by Donncha O Caoimh, an Automattic employee and a core contributor to the WordPress project. Not only is he someone who clearly knows WordPress inside out, he’s also someone I respect.

W3 Total Cache (0.9.1.1)

W3 Total Cache is a contender for the crown. Not a week goes by without me hearing someone influential extolling the virtues of this plugin. It’s worth noting that it has probably the most comprehensive list of features of any WordPress caching plugin. With 7,242 downloads over the last week, its still behind WP Super Cache in terms of usage, but it’s on the rise and clearly a worthy candidate.

W3 TC is written by Frederick Townes, the CTO at the high traffic Mashable website which runs on WordPress. Frederick clearly knows a lot about optimising WordPress for high traffic sites. I’ve also been impressed with the job he’s done as co-host of the WordPress Community Podcast with Joost de Valk.

Hyper Cache (2.7.3)

Hyper Cache is a relatively new plugin, with a smaller market share (2,517 downloads over the last week). It’s not as well known as either of the other two plugins and neither is it’s author, Stefano Lissa. All I know about Stefano is from his About page: he’s a full time developer and analyst.

Hyper Cache was first recommended to me by RT Cunningham from Living in the Philippines. RT’s never steered me wrong, so it would have been worth considering simply from that. The fact that it was written specifically for use on shared hosting was also appealing, as was it’s number one position in a recent benchmarking test.

Note, RT has just pointed out to me that he now recommends the Quick Cache plugin as the ultimate caching solution for WordPress. I guess I’ll have to try that out too – although the testing for this post has been done and the decision made for the moment.

In fact, rather than just considering Hyper Cache as a candidate, I’d decided to just give it a try (without testing the others). However, a Twitter conversation led to me comparing the plugins. A rather lengthy back story section outlines how this came about. You can skip it if you’re just interested in the test.

The Back Story

Having decided to try Hyper Cache, I tweeted the following:

Trying out Hyper Cache instead of WP Super Cache. I was happy enough with WP Super Cache, but heard a lot of good things about Hyper Cache. (source)

sonlinebiz replied with:

@StephenCronin Tried them both but they don’t compare to W3 Total Cache 0.9. Simply amazing and FAST http://strictlyonlinebiz.com/2h (source)

The short link in his tweet goes to his excellent post on speeding up your blog with W3 Total Cache. In response to him I sent a couple of tweets:

@sonlinebiz W3 I’ve heard that Total Cache is great on dedicated hosting – but I’m on shared hosting, which Hyper Cache is geared towards. (source)

and

@sonlinebiz I based my decision to try Hyper Cache on this article, which shows Hypercache beating all comers: http://bit.ly/b8v2AB  (source)

That bit.ly link goes to the benchmarking test I mentioned earlier.

Ozh had also replied and Donncha, the creator of WP Super Cache picked up on the conversation and chipped in (to Ozh, not me) with:

@ozh the benchmark results for supercache in that post look dodgy to me. Configured correctly, it’s Apache mod_rewrite serving cache files.. (source)

I wouldn’t mind knowing more of Donncha’s views on this, because measuring the effectiveness of caching plugins is something I struggled with – but more on that below. I replied with a couple of tweets to Donncha (not worth including here) explaining that I had no particular problem with Super Cache.

Then Ferederick, the maker of W3 Total Cache replied with:

@StephenCronin on the contrary, W3TC has options/benefits for all hosting scenarios (source)

It was great to get a response from Frederick, someone I really respect. Of course, I know W3 Total Cache is going to have benefits even in shared hosting situation – it’s just that I’d heard that its real power was on dedicated servers. I don’t know how accurate that is, but I responded with:

@w3edge Hi Frederick, I know it has benefits in all scenarios, I’ve just heard that you need the persistent object cache to really benefit (source)

And I thought I better point out that I do realise that if W3 Total Cache can’t take advantage of a persistent object cache, then none of the other caching plugins can either (rather obviously), which also limits their effectiveness:

@w3edge I understand that that’s the same for all caching plugins.. (source)

Frederick came back with several tweets:

@StephenCronin it doesn’t really matter what you heard. 🙂 every theme/plugin/server combination varies. (source)

@StephenCronin i wouldn’t have wasted a year working on the plugin if it wasn’t going to help everyone. (source)

@StephenCronin no it’s not, there are more than 70 plugins out there, but there is no other single plugin that has a suite of tools. (source)

This is great: getting direct feedback from the maker of a very popular plugin. However, I already knew that W3 Total Cache is a very good product but I wasn’t going to change my mind, so I tweeted as much:

@w3edge No need to sell me – though I am going to try Hyper Cache dammit 🙂 – I know W3 TC is great. Love the podcast by the way. (source)

Frederick came back with a final tweet:

@StephenCronin don’t care if you use it or not. just the facts. (source [now deleted])

This tweet has since been deleted. I only have a record of it because of a vanity feed in Google Reader (lesson: you can never really delete a tweet).

No matter: I have no problem with the tweet or the fact it’s deleted, but I’ve included it here because it’s important in what happened next, as was my final tweet on the topic (which was trying to lighten the mood just in case people had taken things the wrong way):

So I think I need to put WP Super Cache on one of my blogs, W3 Total Cache on another and Hyper Cache on the last one. 🙂 #wp #caching (source)

Upon reflecting on some key points of the conversation, I actually did change my mind. As Frederick had said:

it doesn’t really matter what you heard. 🙂 every theme/plugin/server combination varies

That’s very true. I was basing my plans on what I’d read when there was indeed likely to be significant variations depending on the environment. The "just the facts" part of his last tweet also got me thinking: what were the facts in this case?

I decided I should follow my last tweet and run all three of the candidates to compare them (although on as test rather than on separate sites).

The Testing Methodology

But first, how can we test the effectiveness of these caching plugins?

I often measure performance using the Debug Queries plugin, but that’s not going to run with a caching plugin installed (well it runs the first time and then it’s results are cached along with the rest of the page).

The benchmarking test mentioned above used Apache Bench, but that wasn’t appropriate for my test. I want to measure the speed of individual page loads, but Apache Bench is more about the effect of lots of traffic on server loads (to my admittedly limited understanding).

My thoughts therefore turned to the Lori (Life of Request info) Firefox extension, a very handy tool that measures how long a page takes to load, including:

• Time to first byte (TTFB) received from the server
• The overall time to load the page
• The size of the page
• The number of requests

The fastest overall time to load is the end result I’m after, but it’s not an effective measurement of a caching plugin because it includes the time to load images and remote scripts and widgets etc, which are outside the plugin’s control.

I *think* Time to first byte is the best indicator of the effectiveness of a caching plugin. If the server is busy running PHP and talking to MySQL, then TTFB will be high. If a page is cached, then it should be served up almost immediately. Of course there may be other things happening on the server which could impact on the results.

I figured that given the lack of an iron clad testing methodology, I’d do the following:

• Run the test without a caching plugin to establish a baseline to compare against.
• Test each plugin twice, once with basic settings, once with more advanced settings (such as compression turned on).
• Within each test, measure results against both the home page and an individual blog post.
• Within each test, measure results both with images and scripts cached and with the browser cache clear.
• For each individual page load, measure both TTFB and the overall time to load.
• Run each page load 10 times and ignore the 2 fastest and 2 slowest results to limit anomalies.
• Monitor the server CPU usage via CPanel to ensure the server is bogged down doing something else.

I’m sure this isn’t the most scientific way to measure a caching plugin, but it’s the best I could come up with and it should at least give an indication of which plugin a particular website loads fastest with.

The Site Being Tested

I decided to use my Jobs in China website for the test. For the single post test, I used the China Is Still The Best Place To Find A Job post.

Note: This site is not well optimised in it’s own right. I have plans to give it an overhaul at some point (minifying and combining scripts, setting up a subdomain as a CDN, etc), but haven’t had time. I was planning to use the caching plugin to cover up the cracks until I had time to do it. Ironically, I could have optimised the site with time I’ve spent writing this post!

The fact that the site isn’t well optimised probably makes it a good test site, as there will be many blogs out there in a similar situation: not particularly well optimised, on shared hosting, etc.

The Results

There are too many test results to present each page load in this post. I’ve therefore included just the averages for each test run. The full results are available on Google Docs.

To fit the summary results in, I’ve separated the results into two tables: one for Time To First Byte (TTFB) and one for the page being fully loaded. First, here are the average times for the first byte to be returned by the server:

Caching Plugin	Home TTFB (Clear)	Home TTFB (Active)	Post TTFB (Clear)	Post TTFB (Active)
No Caching Plugin	1.27	1.20	1.56	1.58
WP Super Cache	0.82	0.84	1.06	1.01
WP Super Cache (with Compression)	0.81	0.83	0.95	1.01
W3 Total Cache (page caching only)	1.51	1.36	1.70	1.70
W3 Total Cache (everything except CDN)	0.72	0.49	0.93	0.46
Hyper Cache (Compression off, min disk on)	0.85	0.60	0.99	0.58
Hyper Cache (Compression on, min disk off)	0.81	0.59	0.87	0.58

Next, here are the average times for the entire page to be loaded:

Caching Plugin	Home Loaded (Clear)	Home Loaded (Active)	Post Loaded (Clear)	Post Loaded (Active)
No Caching Plugin	7.04	5.18	7.88	5.94
WP Super Cache	7.17	5.37	7.36	4.96
WP Super Cache (with Compression)	6.90	3.92	6.37	4.84
W3 Total Cache (page caching only)	7.60	5.96	8.05	5.68
W3 Total Cache (everything except CDN)	6.83	4.54	7.13	4.41
Hyper Cache (Compression off, min disk on)	6.71	3.99	6.63	3.85
Hyper Cache (Compression on, min disk off)	6.24	3.04	6.96	4.05

Analysis

There are a couple of very obvious trends across the board that we should get out of the way first:

• The post takes longer to load than the home page.
• Turning on compression and other advanced options (where available) improved performance across the board.

Those out of the way, here are just a few useful patterns I can see:

• W3 Total Cache actually slowed things down when it only had page caching turned on, although it was one of the better performers when the more advanced options were turned on.
• W3 Total Cache (with advanced options) had the fasted TTFB, which implies that it is the best at serving pages up efficiently and minimising server resources.
• Hyper Cache (with advanced options) resulted in pages being full loaded in the quickest time, even though it wasn’t quite as efficient at at the server end as W3TC. I’m not sure why – perhaps it makes better use of browser caching? However it managed it, Hyper Cache was the fasted across the board for full page loads.
• WP Super Cache was just a little way behind the other two plugins across the board.
• WP Super Cache (with compression turned off) significantly improved TTFB, but had mixed results with the full page load: The home page was slower than not having a caching plugin, although the single post was faster.

As I said above, I’m no caching expert. If anyone reading this can see any other patterns in the results, please let me know in the comments.

The Winner

It depends on what you want!

If you’re after the best optimisation on the server side (ie less time to server up pages, so that it can deal with more requests), then based on the test results, W3 Total Cache is the best choice for you.

If you’re after the fastest loading pages on the client side (ie pages load faster for the user), then Hyper Cache is a better alternative.

Conclusion

The test wasn’t particularly scientific, but it was good enough for me to work out which caching plugin was the right one for me. For the Jobs in China site, I ended up using Hyper Cache.

I’m running W3 Total Cache on this site, because of a recent attack by a bot resulted in HostGator disabling this site. As part of their steps to reactivate the site, they requested I install W3 Total Cache. Because the results are so close and because I wanted the site back asap, I just installed it as requested. So now I just need to add WP Super Cache on a third site…

The biggest take away may be that, as Frederick pointed out, every theme/plugin/server combination varies – so what works best for my particular circumstances may not be best for yours. It’s not hard to run a quick test to compare. Do your own testing before making a choice.

Have an opinion on this test or the plugins mentioned? Please leave a comment below!