Help Center

Menu

Configure an A/B test

The test builder is used to create your A/B test.

All fields except for the description are required for the A/B test to be in a runnable state.

Basic information

The basic setup panel records a name and description for the A/B test.

Test name

(required) Enter a suitable name for your A/B test. This name will be presented on the test management screen, and also in the select list that displays available tests when you connect an A/B test to your page builder block.

Description

(optional) Enter a short description that outlines your test.

Test page

A/B testing can currently only be configured on URLs that are Content page assets stored within the DXP CMS. Other pages (such as Standard Pages) cannot currently define an A/B test.
Page type

Choose where the A/B testing will run.

  • Content page asset (Matrix CMS): The A/B testing is linked to a content asset within the Squiz DXP.

  • Any webpage: (Coming soon) The A/B testing is linked to a generic webpage (can include other pages within the Squiz DXP).

Website domain

Enter the domain of your website where the A/B testing will run. (e.g. www.example.com)

This domain must match the domain of the site where the test is running, but you don’t need to worry about the protocol or any path component (which is ignored).
Content page asset

Displays the connection status of this A/B test. In order to use this A/B test you need to connect it from the content page asset in your DXP CMS website. Will display not connected if this test is not currently connected to a content page.

The A/B test cannot be started until the test is connected and the test is fully configured.

Variants

Content variants

You can define names for your A and B variants, which will be displayed in your A/B testing report. Setup of the A and B variants needs to be completed using the page builder within your connected content page.

Traffic distribution

This option controls how requests to the page will be distributed between the two different variants of the A/B test. The default is to perform a 50:50 split, but you can vary this to match your needs. Vary the amount by moving the slider, or by entering the percentage value directly into the field above a variant.

The percentages assigned to each variant set the probability for a user being assigned to group A or to group B.

The group is assigned when a user first accesses the page containing the variants. A random number between 1 and 100 is generated and the user is assigned to one of the groups based on this number. For example, if you have configured a 10% chance of getting into group A and, when the user visits the page, the random number generated is between 1 and 10 they will be placed into group A. Once assigned to a group the user will always see the same variant for the duration of their session.

This same process is applied to every new user that makes a request.

When a test is connected to more than one content block on a page, the variants are treated as sets with all variant As forming one set and all variant Bs forming another set.

  • If a visitor is distributed to variant A, the page they see will render with variant A active for each content block to which the test has been connected.

  • If a visitor is distributed to variant B, the page they see will render with variant B active for each content block to which the test has been connected.

Success metric

This setting allows you to choose between a page view and a click event to associate with a successful A/B test result.

  • Once a user has been served the A/B test at least once, visiting the success URL or clicking on a desired element at any point during the same session will increment the success metric. There is no minimum or maximum number of navigation steps.

  • The success metric will only be recorded for a user once (per session). If a user performs multiple actions that are designated as a success then only a single click or page view will be recorded for the purposes of the test reporting.

Page view

Success is achieved if the user navigates to the URL specified in the linked Page URL field after seeing one of the test variants.

  • The Page URL must exactly match the destination URL visited by the user (including the protocol and any default page extension) for the test to be marked as successful. The best way to determine this URL is to visit your public site, locate the page you wish to track and copy the URL from your browser address bar.

  • The Page URL must be served by the Squiz DXP via the CloudFlare layer. URLs can only be tracked if they are served by the Squiz DXP.

  • Javascript tracking code is not required for the page views metric.

Click event

Success is achieved if the user clicks on a particular element that you have identified (using the element selector in your Javascript). This requires some Javascript code to be added to your page, which tracks the click event for the element you wish to target.

When you choose to use a click event, a banner will be displayed at the top of the builder page reminding you that you need to install the Javascript for the test to function.

If you click the copy code button on the banner it will copy the Javascript code to your clipboard.

implement js code banner

  • You can tie the click event to multiple elements within your page or across multiple pages (e.g. using a common CSS class) meaning a successful click will be logged if you click on any of these elements.

  • The Javascript tracking code must be included in any page where you are tracking a successful click.

  • Once a successful click is recorded for a user session, subsequent clicks from the same session will not be counted towards the test.

Javascript handler for the custom event

A Javascript handler must be added to your page to handle and register the click event. How this is added will vary depending on the page that you are tracking click events on.

The purpose of this script is to listen for the click event, and submit this as a custom even when a click is detected.

Don’t forget to remove your Javascript code once the A/B test has concluded. It won’t do anything but may result is some errors being logged to your console. 
<script defer>
  (function () {
    const element = document.querySelector('#element_id'); (1)
    const url = new URL('/__dxp/events/custom', window.location.origin);
    if (!element) {
      console.warn('The element was not found. The click event cannot be handled');
      return;
    }
    element.addEventListener('click', async () => {
      await fetch(url, {
        method: 'POST',
        headers: {
          'Content-Type': 'application/json',
        },
        credentials: 'include',
        body: '{"action":"abtest.click","event":{"abTestId":"example-ab-test-id"},"source":"custom"}', (2)
        redirect: 'follow',
      });
    });
  })();
</script>
1 The #element_id needs to be updated to match the selector of element that you are testing.
2 The example-ab-test-id needs to be updated to the abTestId for the A/B test you are linking. If you copy the code using the copy code button on the page banner, this value will be filled in automatically for you. Your AB testing ID can be obtained by managing/editing your A/B test and copying the AB testing ID from the URL. The ID is a GUID, a string of letters and numbers joined with dashes. The URL format is https://dxp.squiz.cloud/organization/<YOUR-ORGANIZATION>/optimization/experiments/<AB-TEST-ID>.

See also