A/B testing of website popups

Created by NotifyVisitors Team, Modified on Fri, 29 Sep 2023 at 07:27 AM by NotifyVisitors Team

In today's ever-evolving digital marketing landscape, effective user engagement is paramount, and one of the most valuable tools for achieving this is through the use of popups. However, optimizing popups can be a complex endeavor, requiring significant resources and expertise. Fortunately, there is a simple and data-driven solution to this challenge – implementing various popup designs and evaluating their performance in real-time to identify the most effective one.

For this purpose, NotifyVisitors offers a powerful tool at your disposal known as A/B testing, which enables you to compare different versions of your popups and determine which one resonates most with your audience. In this comprehensive guide, we will delve into the intricacies of A/B testing for popups, empowering you to harness its potential to boost user engagement and conversion rates.

Understanding the significance of A/B testing for popups

A/B testing in popups involves presenting two or more variations of a popup to your website visitors and measuring their performance to identify the most effective one. This data-driven approach eliminates guesswork and provides concrete insights, enabling you to make informed decisions about your user engagement strategy.

Accessing A/B testing

Follow the steps laid down below to access A/B testing-

  • Navigate to NotifyVisitors Dashboard> onsite>Popup
  • Now here, you can either choose an existing popup for optimization or create a new one. To know more about popups, please refer to article “A Step-by-Step Guide To Create Onsite Pop-Ups
  • To start an A/B Test for an existing popup, click on the three dots available on the specific popup campaign. Then, select "Basic Details" to access the campaign form editor. Within this editor, you'll find the A/B Test option, which you can configure to your preferences.

  • In case you choose to create a new popup, just fill in the name of the popup, choose the label as per your requirement and at last click on update and then you will be navigated to the campaign form editor.
  • Within this form editor you can find the A/B Test option allowing you to customize it according to your testing requirements.

Creating variations and split testing

When you access the A/B testing feature, a menu will appear offering the options to generate up to three different versions of your signup form. There are two methods to achieve this:

1. By duplicating the existing one and making modifications to it: 

You can either duplicate the existing popup and then make the necessary adjustments to it. You can simply click on the copy icon, which will open a dialogue box prompting you to a confirmation message for creating a copy. Once you create a variation, you can modify it as per your requirements.

2. By starting a variation from scratch: 

To create a variation from the ground up, select the "create new Variation" option. This will lead you to a screen where you can name the new variation. After naming it, click "save" to initiate the process and begin crafting a variation entirely from scratch.

Once you have crafted your variations in line with your testing strategy, click on the "save and continue" option to proceed. Next, you can configure and fine-tune the settings for your variation. Upon completion, you can save the variation and return to the original popup by selecting the "save and continue" option.

When you return to the original campaign screen, you can access the A/B test option to view all the variations you have created. Within this section, you can easily edit, copy, or delete a variation to tailor it to your campaign's specific requirements.

Allocation of users

Within the A/B test menu, you have the option to allocate users to each form based on your testing needs. Just click on the settings icon, and you can easily adjust the user allocation by dragging the arrow pointer. After making the necessary changes, click "Update" to proceed. 

For example, you might choose to assign the original form to 25% of users, variation 1 to 20% of users, variation 2 to 30% of users and variation 3 to 25% of users depending on your specific testing strategy.

A/B test analysis

To assess the performance of each popup variation, you'll need to analyze the A/B test results. NotifyVisitors provides an analytics section where you can access A/B test data. Here's how to do it:

  • In the popup campaign interface section, click on the three-dotted button available on a specific popup campaign, then select "Analytics".

  • A new screen will appear where you can click on A/B analytics to view the performance of each variation. You'll see metrics such as unique impressions, unique clicks, click rates and other relevant statistics.

  • Further the A/B analysis includes which variation performed the best. This is referred to as the "winner" variation. Also, you can look for the winning metrics that indicate higher user engagement and conversion rates.


A/B analysis section is only available for Popups using A/B test, i.e. the popup which has multiple variations. For analysis of all other popups this section will not be visible and you will only see daily logs and subscriber logs section.


A/B testing in popups is a powerful strategy for increasing user engagement and improving conversion rates. By following these steps and using the right tools, you can make data-driven decisions to enhance your popups, ultimately leading to greater success in your digital marketing efforts. A/B testing empowers you to refine your popup strategy and deliver the most effective user experiences.

Was this article helpful?

That’s Great!

Thank you for your feedback

Sorry! We couldn't be helpful

Thank you for your feedback

Let us know how can we improve this article!

Select atleast one of the reasons
CAPTCHA verification is required.

Feedback sent

We appreciate your effort and will try to fix the article