Yankee PDP Fragrance Notes πŸ“ Learner (ABT-147 cxp.018)

Owned by Alex Alesio (Deactivated)
Last updated: Jan 05, 2023
3 min read

This wiki is designed to summarize and capture our AB experiment goals, test previews, and results after the experiment concludes.

Results & Recs

Yankee PDP Fragrance NotesΒ πŸ“Β Learner

  • Results: after 21 days of runtime, test results are inconclusive.Β Β 

  • Recommendation: Conclude test and iterate on new test ideas around fragrance notes. By making the fragrance notes more visible, we can provide more information to the users, increase add to bag clicks and conversions.Β See Jira ticket ABT-147Β for additional info.

Test Design

Hypothesis:Β 

By making the Fragrance Notes more visible,
we can provide more information to the users, increase add to cart and conversions.Β 

Metrics:Β 

  • ↑ Add to CartΒ 

  • ↑ Purchases

Β 

v0 Control

v1

v0 Control

v1

Β 

Control modified to have
Fragrance notes expanded by default
(users will not see β€œShow More”)

Β 

Β 

Β 

Β 

Findings & Supporting Data

Β 

Β 

Β 

Β 

Findings:Β 

Data: FullStory Graph of users who clicked β€œShow More” on PDP

  • FullStory data shows that users who click on the β€œShow More” text for the fragrance notes were much more likely to convert than users who just clicked on anything.Β 

  • Users who clicked on β€œShow More” purchased 22.5% from starting on the PDP vs 20.5% for users who just clicked on anything.Β 

  • Increasing visibility of fragrance has the potential to increase CVR.

Β 

Β 

Β 

Β 

AB Experiment Previews:
Screen Shots, Video Capture, & Preview links

Video Capture Preview:

Desktop:

Β 

Β 

Mobile:

Experiment Design:

Hypothesis: Β 

  • By updating or removing the padlock icon we will decrease confusion within checkout, increasing conversions.

Goals:

  • ↑ Conversions

Target: Β 

Results & RECs:

(Results & RECs added Sept15)

Results:

  • After 15 days of runtime the test proved to be inconclusive.

  • No Stat Sig difference between Control and v1 Test Variation.

  • Engagement was dramatically decreased in clicks to the secure checkout icon, with clicks down 40% (100% Stat Sig).

  • For Mobile it was even greater at 44% reduction in clicks.

Recommendation:

  • The experiment successfully reduced friction in Checkout but with no measurable improvement to conversion. Consider rolling out (v1) test variation and continue to iterate and explore test opportunities.

Results by the numbers (AB Tasty Data & Insights)

Β 

AB Tasty Preview Links:Β 

Status & Next Steps:

Idea approved by Yankee BIZ Aug18 (during cxp thurs call)
QA’d by @Alex Alesio (Deactivated) / feedback sent to CXperts Aug24
CXperts resolved AA QA notes
CXperts sent to for approval to Yankee BIZ
Approved by Yankee biz Imran or Natasha
Launched 2022 Aug 30
Test Run Completed 2022 Sept 15
Results (early preview during Sept15 cxp cro call)
Results (full report)