Introduction to A/B testing
A/B testing (also known as split or bucket testing) is used to compare two versions of a content block to determine which performs better.
In the context of your website it allows you to test the effectiveness of changes in your content or design in achieving your goals.
For example, you may decide to change the size and color of a button used to sign up users to a mailing list. A/B testing can be used to help you figure out what works more effectively with your users.
How does A/B testing work?
In the DXP, A/B testing works by allowing you to select a page within your website and set up two variants of a content block on that page.
This is linked to an A/B test which sets the conditions for how and when the test will run, and what constitutes a successful outcome for the test.
You can run a test for all visitors to the page or limit it to visitors in selected segments defined in the Customer Data Platform (CDP). Variant assignment and reporting apply only within the audience you configure.
When the test runs, eligible users that access your website are randomly shown one of the two variants and their actions are tracked against the success metric.
When the test period is complete a report is generated that provides information on which of the variants was more effective.
You can then apply the successful variant and run additional testing to further improve your page.
Audience and segment targeting
Segment targeting limits which visitors can enter an A/B test. You still compare the same two content variants and measure the same success metric; you only change who is eligible and how results are grouped.
Use All visitors when the change should apply to everyone who views the connected page. Use Only segments of visitors when you want to experiment within audiences you define in CDP—for example returning visitors or subscribers—without exposing the test to your full site traffic.
Reporting follows that choice:
-
All visitors — one conversion chart compares variant A and variant B across everyone in the test.
-
Only segments of visitors — up to five CDP segments per test, with a separate conversion chart for each segment.
Configure audience and eligibility in Display options. For when visitors are assigned to variant A or B, see When variant assignment runs.