A/B Testing is NOT Enough!


It’s amazing how many affiliates and advertisers don’t go further than A/B testing – there are literally millions of dollars lost every month because they don’t realize A/B testing is not enough.

After you A/B test you MUST keep testing the winning page for its best variations using multivariate testing.

As we all know, A/B testing can increase your conversion rates and generate more customers, which means more revenue.   In case you don’t know what I mean by “conversions”, I mean “sales”, or “leads”, or any other goal you are trying to achieve on your site.  It’s the juice that makes thing happen.

Multivariate testing can add up to a double digit increase in conversions.

By using A/B testing, you can test different versions of your landing page to see which version leads to a higher conversion rates, but you can’t really test different versions of each section of your page.  It makes total sense to do this, yet 70% of online marketers don’t use multivariate testing after their A/B tests according to recent research.

They are missing out on revenue that wouldn’t cost them very much to capture, if they only would keep the testing process going!

So it’s time to figure it out. You could use a spreadsheet and multiple files, or one of the free tools out there like Google Website Optimizer, but be prepared to devote some serious time to setting things up (read: slicing and dicing your HTML).

A faster, easier way is to use one of the visual test builders that have recently emerged which make building tests as easy as dragging and dropping elements on your page. They take literally only minutes to set up.

Most importantly, make the commitment to start multivariate testing if you haven’t already.  The tools are there to make testing easy now.  Testing ROI has increased dramatically in the last few years thanks to these easy to use tools. (The spreadsheet method is why nobody used to test!)

Ok, now that you’re convinced to use multivariate testing, there are some key elements you need to know how.  It can be difficult to decide where to start, so we put together the most important four elements for you to consider:

Four key elements to test include:

  • Your Heading
  • Special Promotions
  • Call to Action
  • Images

Your Heading – A heading is usually the most prominent element of your landing page.  Most often it’s the first item a visitor notices on the page when deciding whether or not to stay on your site.

The idea of prominent headings is not new; the newspaper industry has been using this strategy for over a hundred years.  To be successful online, you should be constantly testing variations of your headline to find out which version generates the highest conversion rate.

Special Promotions– For many visitors a “special offer” can swing an easier conversion.  You should be testing different offers, prices and promotions for each new offer you build. The only way to effectively do this is with multivariate testing.

Examples of promotions may include testing the following:

  • You can test discounts at two different price levels, such as $10 off $69 and $10 off $79.  Did the $69 offer improve conversion rates?  Was the ROI enough to offset the larger discount?  You can test dollar off versus percentage off, such as $25 off $99 and 25% off $99.
  • Did the customers respond more to the dollar or percentage off?   You can test a message with urgency, such as “offer expires January 31st” or “limited time offer” versus a message with no urgency.  Did the urgency message affect the conversion rate?

Do you know the answers to these questions?  Multivariate testing can tell you.

Call to Action

Your “Call to Action” should always be prominent and clear.  Buttons are the most common method to achieve a call to action.  On a website, the larger an element, the more implied importance it has.  It must be prominent enough to be easily identified, but not so large that it detracts from the headline and other important elements. Have you tested this on your winning A/B page?  That’s where multivariate testing comes in again.

Does your button actually look like a button?  Is it easy for the visitor to tell that this is where they should click?  Is the text on the button a “call to action?”   If you use heat maps, you will be able to see where your visitors are clicking, which is the key to understanding where your visitors click and/or focus the most.

One of the great things about multivariate testing is that you can test buttons with different colors, shapes, sizes and text and find out which one brings your site the highest conversion rate.


Visitors are naturally drawn to images and pictures so make sure all your images relate to and enhance your call to action.

Product images should be clear and crisp.  If you are selling something, it’s obvious a clear image of the item should improve conversion rates over a website with no image or a poor quality one.

Make sure the image isn’t so big that it takes too long too load because slow loading sites lose almost 1% conversion for every second of loading time.

Yes, that’s right, another important point is loading speed, today’s web surfers expect websites to load quickly, make them wait and it will kill you.

Images of people should have clean, clear and easy to see faces. Demographics of your images may vary depending on what type of site you have.  Obviously, if your site sells hearing aids, the images used will be different than if you are selling skateboards.

Whichever your niche, testing all the images to see which ones convert better is the key to making money online.

There are only a few examples of the things you can test on a website – and A/B testing is only the beginning.  Multivariate testing is an even more powerful tool all online advertisers should be using to increase sales.

There is no right or wrong answer on where to start multivariate testing.  The only wrong answer is not to start – remember, A/B testing is not enough!

About the Author

James Borzilleri is the CEO of ConversionDoubler.com, a visual A/B & Multivariate testing solution that lets you create online tests in minutes. James has optimized hundreds of landing pages and knows what works. James is also an author covering Landing Page Optimization, and writes regularly on topics dealing with online marketing and optimization.


What's your opinion?