A/B testing is the process of sending one variation of your campaign to a subset of your subscribers and a different variation to another subset of subscribers, with the ultimate goal of determining which variation of the campaign generates the best results. Therefore, the A/B testing feature allows you to compare different versions of your email campaign to find the best one out of many. 

A/B testing your email campaigns is simple. You can use the email builder to create two different variations of your email and schedule them for a small percentage of subscribers to see which variation performs best. And once the result is generated, it will automatically send it to the remaining subscribers.

Now, if we talk about Split delivery, it is somewhat similar to the A/B testing, but the use case of split delivery is totally different. Split delivery comes into play when you have two or more sets of email content for a single campaign and wish to send them to different subsets of total subscribers. It doesn’t wait for any result, it can be analysed on the basis of stats of the different variations.

For instance, in case you have email content with different offers and you wish to experiment which particular offer(s) elicit a better response, then you can choose split delivery and consequently ascertain which content is helpful for gaining a better conversion rate.

How to perform A/B testing in email campaigns

1. Navigate to NotifyVisitors Dashboard > Campaigns. 

2. Now, select Email, present within the 'Reach Users Via' pop-up.
3. Now, there are five stages to creating an email campaign.
4. In the Recipients stage, fill in all the details like campaign information, recipients, and tracking rules. 
5. Next, in the content stage, you get an option to create a variation for your campaign.
6. We have filled in all the information regarding the sender's name, email address, and subject line.

Here we have created a variation so that we can check which works best for us.

Now, once you have created the variation, the A/B testing and Split delivery option will start reflecting on the same page.

For split delivery, you have to determine the size of the audience for both variations. Here C indicates the Control version and V1 indicates the variation. You can adjust the audience size.

Here, in the A/B test, you will find three sections viz.

• C: indicates the control version i.e. the original version.
• V: indicates the variation.
• R: indicates the remaining section of your audience who will receive the winning version.

Once you have performed A/B testing, the winning version of the two variations viz. A and B, will get dispatched automatically to the chosen percentage of your audience.

What is the difference between A/B testing and split delivery

In the A/B test, all variations clicks and opens are compared within a defined time, and the one with the most is chosen automatically to send to the remaining subscribers. In contrast, with split delivery, the comparison is made manually when the campaign is sent to whole subscribers. Unlike AB testing, Split delivery doesn't wait for any result of the initially sent email. It allows you to split the total subscribers into subsets and send them the desired content.

How to determine the audience for your campaigns

To run an A/B test for the campaigns, make sure you send them to a segmented audience based on their past behaviour on your apps or website.

How to execute the process

1. First of all, create the variants of the campaigns.
2. Determine the campaign reach. i.e. users you want to engage with your campaigns.
3. Choose a testing percentage of users out of the total number of users and send the campaigns to that percentage of users in a specific ratio. We will calculate the winning variation and send the same to the rest of the audience.

This is how you can perform A/B testing and split delivery for email campaigns in NotifyVisitors.

How to analyse A/B testing stats for email campaigns

A/B test statistics for email campaigns can be useful to discern information such as how many customers opened and clicked on each variation of the campaign, how many unsubscribed from receiving future notifications etc.

NotifyVisitors makes it significantly convenient for you to review the A/B test statistics for any email campaign, since you can comfortably check analytics such as Winner, Winning Metric, Unique Opens, Unique Clicks etc. for the dispatched variations, provided in a well organized manner, under the A/B testing tab. By means of perusing the available analytics, you can easily ascertain exactly which variation performed better and was thereby dispatched to the majority of your customer base.

To understand the entire process of A/B test stats of an email campaign, check out our article Review A/B test statistics for email campaigns.