November 14, 2013

AB Testing: The Why & How

Posted by Faith Albert Category IconConversions & UX Category IconTechnology

Photo courtesy likethebirds, Flickr
Photo courtesy likethebirds, Flickr

A/B testing, CPC, SEO, SEM — so many buzz acronyms, so little time. Recently, marketers have been tossing around the words A/B testing about as much as the average person says thank you. There is a lot of hype, but what is this tool good for and where do you get started? Think back to middle school science class. You likely learned about conducting experiments. A/B testing is no different than a simple science fair project. In short, A/B testing is the act of putting a control against a variable in the hopes of learning which option helps your company reach its goals. Goals can vary, but almost always pertain to the call to action: donate, buy, sign-up, give feedback, etc. When someone reads your email or visits your website, what do you want them to do, what do you want them to experience? Once you know the answer, you can begin testing.

Testing usually consists of digitally publishing one or more variations of the same feature. Whatever variation draws the most clicks or gets the most attention wins. This is decision making made with hard data, not opinion. Email subject lines are usually what companies like to test most. However, try thinking outside the box. What company’s websites do you find visually and functionally appealing? Why do you like it? Do fields auto populate when you check out? Can you sign in with Facebook? Do you prefer clean pictures up top and writing underneath, or do you prefer a mix? Ask colleagues the same questions. In the end, you will have a huge list of things to test that will go far beyond headlines.

Let the experimenting begin. It is important that you test the variations side by side in the same environment. For example you wouldn’t test the effects of vinegar on plant growth during two different times of the year. You wouldn’t know if the results were a product of the time of year or the vinegar. Likewise, if you want to test whether the words “buy now” or “let’s get started” lead to more conversions, they need to be tested simultaneously to (ideally) the same relative audience.

Your results may not be what you want to hear. A giant button that is black and yellow and says “SIGN UP” could lead to more conversions, but at what cost? It’s important to remember your goal for the testing may be to get as many conversions as possible, but you will need to weigh that against company branding and design to ultimately decide on making it a permanent addition to your communications and site. You don’t always have to let the statistics win out. A/B testing will give you a handle on what is happening when people visit your site or read your emails. It is up to you to decide what works.

It’s pretty simple to see why companies are using A/B testing. Software manufacturers specializing in only A/B testing are popping up everywhere. Optimizely is one of the leaders in the industry and promises to make the process as simple and easy as possible. So why not give it a try. Who knows? Unlike your 7thgrade science fair, you might win first prize.