View A/B test results

A/B testing results

A/B test results

Live A/B test results are displayed within the test’s configuration page.

  1. Navigate to the page from DXP  Optimization  A/B Testing, then use the filters or a search to locate the test.

  2. Click on the specific test entry to open the test’s configuration page.

The Results section displays (below the Summary section).

When the test targets All visitors, one aggregate conversion chart compares variant A and variant B. When the test targets Only segments of visitors, see Segment-targeted results.

results

Further information is presented when a pointer hovers either of the results bars.

For example, a running A/B test presenting results:

variant a and b no hover

Hover the pointer over the Variant A results bar, and a pop-up presents:

variant a on hover

Hover the pointer over the Variant B results bar, and a pop-up presents:

variant b on hover

In both pop-ups,

  • the Total visitors amount is the number of times the variant page was viewed; and

  • the Page views is the number of times the success metric was achieved.

A/B test results are based on unique sessions.

Also, A/B test results presented in the DXP Console are updated approximately every eight hours.

If an A/B test is connected to multiple content blocks, the test results represent the success of a variant set in driving the stated success metric.

They indicate whether variant A set or variant B set is more successful.

You cannot measure how one of the content blocks performs against another. To test at that level of granularity you must run an A/B test specifically on that content block.

Segment-targeted results

When you configure Only segments of visitors in Display options, live results use the same Results section on the test configuration page.

You can attach up to five CDP segments to the test. The Results section shows a separate Conversion rate chart for each segment instead of one aggregate chart.

segment results charts

Further information is presented when a pointer hovers either of the results bars; pop-up behaviour is the same as for tests that target All visitors (see above).

Each chart is labeled with the segment name and compares Variant A and Variant B for visitors in that segment only.

When the test includes multiple segments, read each segment chart separately to decide which variant performed better for that audience.

Export a PDF report

Once your A/B test is finished you can choose to save your test results.

Click the Download report button to download the A/B test results as a PDF.

The report includes a summary, basic details on the test and the success metric.

The report presents a comparison of the conversion rates for the two variants that were tested.

When the test targeted Only segments of visitors, the PDF includes a conversion rate comparison for each selected segment, with variant detail for each segment.

segment pdf report 1
segment pdf report 2

When the test targeted All visitors, the PDF presents one comparison chart for the two variants.

pdf report

The Download report button presents immediately an A/B test is finished.

When an A/B test is finished, all the data generated by the test has been processed by the test. However, in some cases, the metric service is still processing some metrics even though the A/B test itself is done.

In such a case, A PDF generated before the metric service is done processing will present different information than a PDF generated after the metric service is done.

Implementing the findings

The results of the A/B test will provide you with information that can be used to decide if you wish to implement either of the variants, or to perform a new test.

At this point you could choose to implement variant A on your site, or you could run a new A/B test comparing variant A to a further variant to further optimize the experience.

The basic steps to implement the successful variant on your site are summarized as follows:

  1. Open the content page asset that contains the A/B test.

  2. Add a component to the content page that matches the A/B test component.

  3. Copy over the content from the test variant that yielded the desired result.

  4. Delete the old content component that contained the A/B test.