20 January 2016

An Introduction to A/B Testing (Part 1)

Continuous improvement is better than delayed perfection.
– Samuel Clemens (a.k.a. Mark Twain)

Introduction

Sold/won.
Topline.
Revenue.
Your data tells a story. Listen. Sell more.™
The revenue opportunity with the best ROI for any organization is increasing the win rate of existing leads/prospects.
A/B testing is where you start. It is low cost and relatively quick.
The communication your organization shares with prospects has the goal of increasing conversions.
A/B testing helps you determine how to improve individual pieces of communication media to increase conversions and achieve your organization’s objectives.

What is A/B testing?

A/B testing is comparing two versions of a piece of media (an email, a web page, a digital advertisement, etc.) to see which one performs better.

The performance of each piece of media is compared by showing the two variants to similar users at the same time.
The one that gives a better statistically-significant conversion rate, wins.
A-B testing examples
typeAB
button color
buttons vs. linksVisit our site
graphical vs. text element

visit us
subject lineThree-day Sale25% off this weekend
senderFrom: SalesFrom: Bob
PersonalizationDear CustomerDear Sue
menu order
  • Home
  • Buy Now
  • Contact Us
  • Home
  • Contact Us
  • Buy Now
It is best to use caution when generalizing the results of one A/B test to all of your emails. All the result tells you is that for this particular scenario one option outperforms the other. When extending the learning to prescribe changes in other emails, A/B tests should be conducted on those emails before the change is made globally.

Why should my organization do A/B testing?

Try this – you’ll be amazed
Construct a table like the following for your first three emails.
Email #Opened?Win Rate
1Yes25%
1No10%
2Yes30%
2No8%
3Yes32%
3No5%
Enter your values below
Email #Opened?Win Rate
1Yes %
1No %
2Yes %
2No %
3Yes %
3No %
What you will discover is that the customers who open your email are more likely to convert to a win.
You can sell more simply by selling smarter.

Cost & Time

A/B testing a particular piece of media is free (you’re already paying for the email marketing automation system, website, etc.).
A/B testing a single piece of media generally takes little time to set up and start.
The other things you can do to increase sales — Price (adjustment/discount), Promotion (advertising), People (hiring & training), Place (new location(s)/relocation(s)), etc. — tend to have meaningful OPEX/CAPEX costs. They also can take meaningful time to implement.
Low resource utilization & quick vs. high resource utilization & slow — if you want more conversions, A/B testing is the logical place to start.

Segmentation & Lead Scoring

How customers interact with a piece of optimized media allows you to group them into segments, allowing more use-case/persona specific messaging.
With an ensemble of responses to optimized media, you can group prospects into clusters, which are very, very valuable for lead scoring.
With reliable lead scoring, your sales teams will be able to prioritize higher-performing clusters. Your revenue will increase without additional investment — a ROI win!

What can be tested?

Email

  • Subject lines
  • Calls to action (CTA)
  • CTA colors
  • Cadence (the timing between emails)
  • Omission (whether a particular email in the sequence matters)
  • Sending day of week
  • Sending time of day
  • Deliverability

Web

  • Landing pages
  • Sign-up pages
  • Menus
  • CTAs
  • Site navigation elements
  • Responsive design layouts

Display advertising

  • Flash vs. animated GIFs
  • CTAs
  • Destination landing page
  • Dayparts
  • Behavioral/demographic targets

Search engine marketing

  • Headlines
  • Display URLs
  • Descriptions
  • Extensions
  • Day parts

The next part of this series "Designing an A/B Test" will published in the next few days.

No comments: