Yankee Checkout UI Simplification Padlock (Security Shield) Icon (ABT-108)

This wiki is designed to summarize and capture our AB experiment goals, test previews, and results after the experiment concludes.

Experiment Name:
new-22-008 Checkout UI Simplification | Padlock / Security Shield Icon Swap (Yankee) [ODYi-1528]

AB Experiment Previews:
Screen Shots, Video Capture, & Preview links

Video Capture Preview:

Desktop:

AB 008 Test - Checkout UI Padlock to Shield DESKTOP ( 2022-08-24 dF0iQIRhao ).mp4

Mobile:

Experiment Design:

Hypothesis:  

• By updating or removing the padlock icon we will decrease confusion within checkout, increasing conversions.

Goals:

• ↑ Conversions

Target:  

• Yankee DTC website ( YankeeCandle.com )

• Checkout Pages

• All Users / All Devices

Results & RECs:

(Results & RECs added Sept15)

Results:

• After 15 days of runtime the test proved to be inconclusive.

• No Stat Sig difference between Control and v1 Test Variation.

• Engagement was dramatically decreased in clicks to the secure checkout icon, with clicks down 40% (100% Stat Sig).

• For Mobile it was even greater at 44% reduction in clicks.

Recommendation:

• The experiment successfully reduced friction in Checkout but with no measurable improvement to conversion. Consider rolling out (v1) test variation and continue to iterate and explore test opportunities.
  

Results by the numbers (AB Tasty Data & Insights)

AB Tasty Preview Links: 

• v1 Shield Icon: https://protect-us.mimecast.com/s/8So-CgJyM7Tl57Y5XF2rPbo?domain=app.abtasty.com

Status & Next Steps:

• Idea approved by Yankee BIZ Aug18 (during cxp thurs call)
• QA’d by Alex Alesio (Deactivated) / feedback sent to CXperts Aug24
• CXperts resolved AA QA notes
• CXperts sent to for approval to Yankee BIZ
• Approved by Yankee biz Imran or Natasha
• Launched 2022 Aug 30
• Test Run Completed 2022 Sept 15
• Results (early preview during Sept15 cxp cro call)
• Results (full report)