A/B testing

Ben Logan - Published: 21st Apr 2009 12:48 GMT

In this article we look at "A/B" or "Split" testing and explore the basics around what this is and some tips on how you can improve your A/B testing.

What is A/B testing?

A/B testing is not a new concept and has been used for a long time in offline marketing particularly with direct mail.

The aim of A/B testing is to compare two different versions of a page or a page element such as a heading, image or button in order to see which elements are engaging users to make them respond and to improve conversion rates.

A/B testing is also referred to as "Split testing" as the differences between the two pages are split or alternately served to different users.

Why use A/B testing?

Rather than debating with your colleagues about which call to action or button will increase the number of sales or conversions on your site, you can run a scientific test which will calculate the numbers for you. Often the results are surprising but the numbers don't lie!

Advantages to A/B testing

• Low cost to deploy as requires only a simple change to an existing page
• It measures the actual behaviour of your customers under real-world conditions
• The process is not visible to most of your users
• Can solve internal debates about calls to action and design choices

Disadvantages to A/B testing

• More suited to high traffic sites
• Requires knowledge to setup the campaign and interpret the results
• In some cases results need to be gathered over a period of several months
• Is more suited to sites with clear measurable actions such as a newsletter sign up and actual sales
• You can only test two pages at a time unlike multivariate testing

Examples of A/B testing tools

We have come up with a brief list of companies that provide tools for A/B testing:

• Google website optimizer
• Maxymizer
• Omniture

back to top