AB Testing & Experimentation Update (2022-12-15)

Owned by Alex Alesio (Deactivated)
Last updated: Dec 15, 2022
3 min read

AB Update Email || Subject: AB Testing & Experimentation update | Sent 2022-12-15

Hello Team,
We wanted to provide a quick update regarding our latest AB Experiment results.  

Special Thanks to our AB Partners!
We’re already seeing some impressive numbers including a $598.92K uplift resulting from Yankee & Graco’s experiments which we kicked off in July of this year. As you can see your support goes a long way—keep those ideas coming!
  

⭐️ AB Winners & Trending Tests    

Yankee Global Carousel Stripe ⭐️ Winner 

  • Results: after 22 days of runtime, v2 (Carousel Banner Removed) is a Winner.

  • Recommendation: While v2 was the winner (Banner Removed), we recognize the business value provided by the site stripe and have plans to continue testing and iterating on the design of the component moving forward.  

Yankee NAV L1 Category copy New -vs- Shop-By ⭐️ Winner

  • Results: after 29 days of runtime, v1 is a Winner with the greatest potential revenue lift across all 3 variations.   

  • Recommendation: Implement v1 “New” L1 category copy as v1’s simplicity outperformed today’s “Shop By” resulting in a combined increase in Transaction Rate & AOV Lift metrics. 

  • Additional info: Odyssey enhancement VOC ticket logged: ODYI-1807
    Wiki Page / Jira Ticket ABT-118 

Graco Checkout CC Credit Card elevate placement 📈 Trending

  • Results: this test has been running for 14 days and it looks like v2 (Credit Card elevated to first position and preselected) is close to Winning…  

  • Recommendation: confirm v2 as a winner by EOW and consider implementing the v2 variant where we elevated the Credit Card payment option to the first/top position in addition to keeping the CC payment method radio button is selected by default. 

  • Additional info: Odyssey enhancement VOC ticket logged: ODYI-1808
    Wiki Page / Jira Ticket ABT-131 

📝 Test Learners  

Yankee PDP Fragrance Notes 📝 Learner

  • Results: after 21 days of runtime, test results are inconclusive.  

  • Recommendation: Conclude test and iterate on new test ideas around fragrance notes. By making the fragrance notes more visible, we can provide more information to the users, increase add to bag clicks and conversions.
    See Jira ticket ABT-147 for additional info. 

Graco NAV L1 Shop-All / Journey-Page 📝 Learner

  • Results: after 33 days of runtime, test results are inconclusive.

  • Recommendation: End test and implement based on preference.
    See Jira ticket ABT-155 for additional info.

Yankee Global Auto-Ship Toggle Location 📝 Learner

  • Results: after 13 days of runtime, test results are inconclusive.

  • Recommendation: end test and move on to more fruitful experimentation opportunities.
    See Jira ticket ABT-115 for additional info.

🚀 Launching Soon!    

Yankee Cart Page Rewards Placement

Graco PLP Hide Retail Exclusives

Graco Cart Display of Applied Discounts

Have a test idea? Submit your experiment hypothesis today @ Bit.ly/abRequest or Slack us!  
We’re happy to discuss test ideas and address any questions you may have!    

Have a great day!
- @Alex Alesio (Deactivated)