Page Comparison

This wiki page is designed to summarize and capture our AB experiment goals, screen shot previews, and results after a test concludes.

Experiment Name: new-22-008 Checkout UI Simplification | Padlock/Security Shield Icon swap Swap (Yankee) [ODYi-1528]

AB Experiment Previews:
Screen Shots, Video Capture, & Preview links

Video Capture Preview:

Desktop:

AB 008 Test - Checkout UI Padlock to Shield DESKTOP ( 2022-08-24 dF0iQIRhao ).mp4

Mobile:

Experiment Design:

Hypothesis:  

• By updating or removing the padlock icon we will decrease confusion within checkout, increasing conversions.

Goals:

• ↑ Conversions

Target:  

• Yankee DTC website ( YankeeCandle.com )

• Checkout Pages

• All Users / All Devices

Results & RECs:

(

Results & RECs added Sept15)

Recommendation:

• ( pending )

Results by The Numbers: AB Tasty Data

Results:

• After 15 days of runtime the test proved to be inconclusive.

• No Stat Sig difference between Control and v1 Test Variation.

• Engagement was dramatically decreased in clicks to the secure checkout icon, with clicks down 40% (100% Stat Sig).

• For Mobile it was even greater at 44% reduction in clicks.

Recommendation:

• The experiment successfully reduced friction in Checkout but with no measurable improvement to conversion. Consider rolling out (v1) test variation and continue to iterate and explore test opportunities.
  

AB Tasty Preview Links: 

• v1 Shield Icon:
  https://protect-us.mimecast.com/s/8So-CgJyM7Tl57Y5XF2rPbo?domain=app.abtasty.com

AB Tasty Preview Instructions:

1. Close other tabs that are the main site domain

2. Click the link above, which will take you to the test preview landing page (so not in private browser to allow cookies)

3. Click the test preview link and verify you are getting the variation

4. Verify on desktop and mobile breakpoints (collapsed desktop browser is fine)

Status & Next Steps:

•  Idea approved by Yankee BIZ
  Aug18 (during cxp thurs call)
•  QA’d by Alex Alesio (Deactivated) / feedback sent to CXperts
  Aug24
•  CXperts resolved AA QA notes
•  CXperts sent to for approval to Yankee BIZ
•  Approved by Yankee biz Imran or Natasha
•  Launched 2022 Aug 30
•  Test Run Completed 2022 _____ Sept 15
•  Results (early preview during Sept15 cxp cro call)
•  Results (full report)