How do i A/B test Recommendations?

With some extra customization detailed in this page, you can turn Klevu Recommendation banners on and off according to your Google Tag Manager A/B test.

Prerequisites

• You already have Google Tag Manager created.
• You are already using Google Optimize for A/B Testing.
• You have already configured Klevu Recommendations for your store.

JavaScript Modifications

By default, Klevu Recommendations will power up as quickly as possible. However when using A/B testing via GTM we need to make sure we wait for Google to tell us whether Recommendations should be displayed or not for that customer.

During Klevu Recommendations setup, you will have added a snippet of JavaScript to initialise Klevu. This needs to be modified in a number of places to wait for GTM to see if this particular customer should see Recommendations or not.

Please read the inline comments within the following link for guidance on how to add this to your store, click here for the Klevu Recommendation A/B Testing code

Google Tag Manager Configuration

Next we need to create a new experience within Google Optimize which will control whether or not a particular customer should see the Klevu Recommendation banners.

• Go to your Google optimize account.
• Click on Create experience button.
• Name the experience Klevu Recommendations A/B Test
• Enter the link to your store homepage.
• Click on the Create button.
• In the targeting and variants click on the Add variant button
  • Name the Variant Enabled Recommendations.
  • Click on the Done button.
• In the Variants section find the Enabled Recommendations variation and click the Edit button.
  • In the Edit mode, click on < > in the top right and select Global JavaScript.
    • In the popup window enter “window.klevu_isRecsABTestActiveForCustomer=true;”
    • Click the Apply button.

Next we need to connect the experiment to Google Analytics.

• In the Measurement and objectives section, click Link to Analytics and select your GA account.
• In the Objectives section click on the Add experiment objective.
  • Select Choose from the list dropdown.
  • Select Revenue as your primary objective.
    • We also recommend you repeat the process to add Transactions and Pageviews as your secondary objectives.
• Scroll down to Settings and click Check Installation to ensure everything is OK.

You are all done! You can preview your variants by hitting the Preview link. The original variation should not show Klevu’s product recommendations and the Enabled Recommendations variant should show them.

When you make sure everything is ok click the Start button located in the top right corner.

Viewing Google Optimize reports

Enter your Klevu Recommendations A/B test and click on the Reporting tab to see the A/B test performance.

Viewing Google Optimize data in Google Analytics

Go to your Google Analytics account → Click on Behavior → Experiments → Select Klevu Recommendations A/B test and you will see your experiment results.