...
Experiment | Experiment Name | Test Details | Results & Experiment | Next Steps & Action Items: | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Winner ⭐️ | Quantity Selector removal on HP PLP Hypothesis: By removing the Quantity Selector on the PLP, we will reduce user friction and hesitation on the PLP, increasing PDP visits, add to bag clicks, and conversions. | ↑ PDP Visits Test Timing: | Results (the verdict): After 20 days of test runtime, v1(Quantity Selector Removed from Product Tiles) is a Winner! | RECs & Next Steps
Additional Info:
| ||||||||||||||||
Learner | PLP Star-Rating # Reviews presentation Hypothesis: By adding the number of reviews underneath the stars on products within the PLP, we will illustrate product popularity accurately, decreasing user hesitations, increasing “Add to Bag” CTA clicks and conversions. | ↑ Add to Bag CTA Clicks Test Timing: | Results: Test ran for 14 days and the results are inconclusive. No stat sig impact on order confirmation page visits or other checkout page visits between either variation. No differences between mobile or desktop users. | No next steps required—learner experiment. Recommendation; conclude the test and move on to more fruitful tests. This is a good reminder that users may say one thing but do another, in that reviews are important to them, but maybe not on the PLP. Additional Info: | ||||||||||||||||
Winner ⭐️ | PLP Filter Order (Yankee) Hypothesis: By changing the order of the filters in the sidebar, we can help users find their preferred product more easily, increasing PDP visits, add to bag CTA clicks, and conversions. | ↑ PDP Visits Test Timing: | Results: the test ran for 22 days and v1 (Filters Reordered) is a Winner! Result Details: v1 Filter Interactions had a stat sig uplift, increasing by ~15% with 100% stat sig. Although Filter Toggle Interactions may have decreased (not stat sig), all other filters received uplift of at least 6%, most notably Fragrance Type & Color which increased by ~20% with 100% stat sig. | RECs & Next Steps:
Additional Info: | ||||||||||||||||
Winner ⭐️ | Add-to-Bag CTA Color update across Yankee Hypothesis: By changing from a “ghost” button style to a filled button style with a primary color, we can increase Add to Bag CTA clicks on the Homepage / PLP and thus conversions. | ↑ [Add-to-Cart] Clicks Test Timing:
| Results: Green wins! | RECs & Next Steps:
Additional Info: | ||||||||||||||||
Learner | Cart Page Minimized NAV Bar (Yankee) Hypothesis: By minimizing the nav bar by default, we minimize distraction and focus users to the checkout process, thus increasing the number of users continuing to checkout and transacting. | ↑ [Cont-to-Checkout] CTA Test Timing: | Results: Minimized Nav (v1) showed increased CTA Clicks [Continue to Checkout] and slightly reduced bounce rates. | RECs & Next Steps:
Additional Info: | ||||||||||||||||
Learner | Checkout Guest CTA (Yankee) Hypothesis: By changing the color of the primary Sign-In CTA for guests in the checkout flow, we will increase conversions via Guest checkout. | ↑ [Cont-as-Guest] Clicks Test Timing: | Results: After 21 days of runtime, this test is inconclusive. Both Mobile & Desktop do not have stat sig difference in behavior for primary goal (Clicks to Checkout as a Guest CTA) or macro goal (Order Confirmation Page Views). | (No immediate follow up—test results were mixed) Potential next steps: Consider additional testing on Checkout to understand how Loyalty Messaging on Checkout can incentivize users to Sign-In to generate Rewards Points. (This test is listed in our experimentation IDEAS area and is under consideration for future testing). | ||||||||||||||||
Learner | Global Bag to Cart (Yankee) Hypothesis: By changing the bag icon and copy “Bag” to a cart icon and copy “Cart” within the global nav header, we can increase cart clicks, engagement, and conversion. | ↑ Conversions Test Timing: | Results: After 14 days of runtime, the results are mixed. Engagement: we see v1-Cart driving a 100% stat sig increase in global cart icon clicks by about 3.2%, leading us to believe it may be more visible or interesting to users. | (No immediate follow up—test results were mixed) Potential next steps: Consider prioritizing tests around Mini-Cart functionality to validate learnings of this test and revised hypothesis: … mixed results may be a result of users intending to engage the mini-cart (available with add-to-cart clicks), thus the increase in clicks to the Cart icon, but were potentially frustrated in that lack of behavior. | ||||||||||||||||
Learner | PDP Add-to-Bag CTA Color Swap (Yankee) Hypothesis: By changing the add-to-cart button color, we can increase the number of add-to-cart clicks and conversions. | ↑ [Add-to-Cart] Clicks Test Timing: | Results: Original CTA color outperformed both variants with 98% stat sig for [Add-to-Bag] CTA clicks. | (No immediate follow up.) CTA color updates from this test were merged into separate ticket. RECs & Next Steps: Merged related learning regarding ATC [Add-to-Cart] CTA Color with Winning test 004. Learnings merged into voc ticket:
|
...