New to digital marketing and stick to one design/email/landing page?
You seriously need to AB test your stuff.
Are you confused?
If yes, let me help you solve the concern with this article!
What is AB Testing in Digital Marketing?
AB Testing is a way to measure the performance of two versions of a web page or an email against each other.
The version that outperforms the other one wins, and you keep it.
Let’s Get Technical
It’s just like testing your blood! You get two samples, A and B. Sample A & B are similar but not the same.
Sample A contains a drug, and Sample B doesn’t.
You give the drug to 1000 people, then check how many are alive after one month.
If more people survive from sample A than sample B, then you will use that drug in the future.
AB Testing is just like that but with HTML code instead of drug.
In digital marketing, you test two versions of a webpage or email and see which one converts better than the other.
Email AB Testing
In email marketing, there are many versions that you can create to test your emails.
For Example, You want to send an Email Marketing campaign about some new Product ABC, then for version 1of that campaign. You can test the Subject Line, for version 2 you can test the body of the Email, or you can test both.
As an example, let’s say there are two versions of your Email:
Version 1: Hi <<first_name>>! Our factory is overstocked with Product ABC until next Monday at 10 am. Don’t forget to grab yours today!
Version 2: Hi <<first_name>>, did you know our factory is overstocked with Product ABC until next Monday at 10 am? Don’t forget to grab yours before it’s too late.
Both of the emails are similar but not the same – they have changed a few words around and added one extra line.
You might be wondering…
Which Email will convert better? Or Is there any difference between both emails?
That’s what AB testing is for! And that’s where the statistics part comes into play.
Normally you would send an email to 100 people or more and then compare the data of which Email won.
For example, we sent an email to 100 people and then tracked which version of the Email had a higher open rate.
Suppose
In this case, version 1 has a higher open rate than version 2.
It means that more people opened the first one.
You can do AB testing with Facebook ads / Google ads/YouTube ads, landing pages, email sequences, and much more!
Landing Page AB Testing:
Let’s take another example. Let’s say you have a landing page that converts at 46%, but you want to increase that.
You can do this by AB testing your landing pages.
There are many ways in which you can do so, but the most popular one is changing your Call to action button color.
For Example, you want to know whether the “Register Now” or “Sign up” button converts higher? Hence you can change the button text of both pages and then track which page had a better conversion rate.
Suppose: In this case, version 2 (register now) has a higher conversion rate than version 1 (sign up).
It means that more people are registering after clicking the “Register Now” button.
Now you know what AB testing is in digital marketing!
Software AB Testing:
Softwares are very important in digital marketing and help you to run AB tests on your marketing campaigns.
For example, with Adwords, Google’s Conversion tracking software can be used as a tool to perform an AB test.
AB testing is a great way to increase your conversions and revenue, but make sure you don’t overdo it with AB testing.
A good rule of thumb I follow is to only have one variable in an AB test for a given time period (for example, if you are testing a button color, don’t test it against a popup, text, and so on).
If you do that, then the variable in your AB test won’t have enough significance to get statistically valid results.
Remember, when you are running an AB test, make sure you are setting up a hypothesis beforehand and go with your gut feelings.
If the data shows that your AB test is working, then it is probably right to go with one version and if you are not getting valid results – stick with the original until you find out why it didn’t work.
A good rule of thumb I follow is to only have one variable in an AB test for a given time period (for example, if you are testing a button color, don’t test it against a popup, text, and so on).
If you do that, then the variable in your AB test won’t have enough significance to get statistically valid results.
How does it work?
In simple words, it’s a way to check which button color, headline, or image works best for you.
I will break this into two parts
- Testing Apps & Software.
- Statistical Significance AB Tests.
Testing Apps & Software
There are plenty of ab testing apps & software tools that you can use.
Many are free, while some charge a little money.
Statistical Significance of AB Tests
In simple terms, statistical significance helps us understand whether results are real or just fluke. It is also known as “Hypothesis Testing”.
In the case of AB tests, it helps us determine whether a change that we made into the page or any other element caused an increase in conversions.
If you run AB tests on your site, then it is very helpful to know about Statistical Significance.
The Null Hypothesis and Alternative Hypothesis
In the case of AB testing, null hypothesis reveals no change in the conversion rate. Whereas, alternative hypothesis shows change in conversion rate.
The following points should help you understand this better:
- In the case of AB testing, assume that your original button color has a 20% conversion rate.
- If, after changing the button color to green, the conversion rate increases to 25%, then this means that there is a change of 25% – 20% = 5%.
- From this data, you can conclude that changing the button color increased conversions.
- In case you run another AB test and find out that after changing the background color to red, the conversion rate increased to 25%, then this means that there is a change in conversion rate of 25% – 5% = 20%.
- You can see that changing the background color increased conversions by a much higher percentage than changing the button color.
- This shows that you need to have more samples for the AB tests.
- In the case of large sample size, since both the samples are tested on different types of elements (button color and background color), then any change in conversions gets divided into smaller changes caused by each element (button color or background color).
Statistical Significance
This brings us to the concept of Statistical Significance explained below.
Statistical significance is a level of confidence by which you can determine whether the result of an AB test is statistically significant or just random.
It’s important to know what we are trying to achieve with statistical significance. We want to answer one question: Is there a meaningful difference between my control and my experiment.
This helps us determine whether the changes that we made into our experiments actually increased the conversion rate or just happened to be a coincidence.
Here is how you can find this out.
- Pick a level of confidence.
- Assume that for your control, you got 20 conversions in 100 visitors.
- Now, run the test for a week or two and find out that your experiment has 25 conversions in 100 visitors.
- If you pick 95% confidence, then this means that the difference between 20 and 25 is not significant. In fact, it may be just random variation caused by “noise” in the data.
- So, you can see that even if your test was successful in increasing conversions (25% vs 20%), it would not be significant at a 95% level of confidence.
- However, this does not mean that there is no difference, and hence you should ignore this result.
- You need to run the test for a longer duration or with a higher number of visitors to determine whether this change is statistically significant.
- For 95% confidence, you need to have at least 14 conversions in each sample (that’s 100 divided by 95%), and that takes time. If you see that your confidence level goes up every day as more people convert, then you can end your test early.
I know this is a hard pill to swallow, but we have to understand these things if we want to do some great AB tests in our digital marketing career!