Hybrid A/B Testing Analyst (Product Analyst, A/B Testing) at Levi Strauss & Co.

Calling all originals: At Levi Strauss & Co., you can be yourself — and be part of something bigger. We’re a company of people who like to forge our own path and leave the world better than we found it. Who believe that what makes us different makes us stronger. So add your voice. Make an impact. Find your fit — and your future. 

Levi Strauss & Company is pursuing an AB testing strategy to achieve a best-in-class eCommerce experience. This is an exciting opportunity for a business-minded, data savvy superstar: the successful candidate will play a key role in accelerating the growth of Levi’s eCommerce businesses via site testing. 

In this role, you will have the opportunity to work across global cross functional strategic and marketing teams to analyze site and app A/B testing efforts.  This includes gathering and analyzing information, testing hypotheses, analyzing tests results, and communicating recommendations based on results.

You will develop analytics solutions to improve product visibility, identify product friction, elevate the Site and App experience for levi.com, and partner with product, data science, and business teams to help improve shopper experience.

About the Job 

• Create dashboards and track KPIs, adoption, and business impact of A/B tests.

• Partner with product team members on planning, goal setting and product prioritization.

• Proactively identify data driven opportunities to improve shopper experience and partner with cross functional teams on implementation. 

• Able to detect frustration, abandon, or engagement. Highlight exits or bounces in a consumer’s navigation.

• Provide pre/post guidance and analytics review to test owners based on KPIs.

• Communicate findings and recommendations in a way that is clear and actionable for employees at all levels.

• Make recommendations and provide launch strategies based on their analyses to increase the feature’s profitability. 

• Promote a culture of excellence, data driven discussions, healthy skepticism, knowledge sharing, expanding team’s skill set, teamwork, and focus on Levi’s growth.

Key Responsibilities:

(30%) Site and App Testing Strategy & Execution:

• Work with stakeholders to understand business needs, gather test requirements, KPIs, gather qualitative and quantitative data, form hypotheses, develop predictions, and define success metrics

• Identify implications and underlying assumptions of each test to determine how each variable affects tests and develop complete problem statement summarizing the hypotheses and expected results

• Perform statistical significance sample size calculation based on primary and secondary metrics for estimated test duration based on statistical confidence levels.

• Ensure proper tracking requirements are in place for tests measurement during test planning phase

• Assist in ensuring the correct set up for tests and personalized content experiences in the A/B testing tool to capture the data accurately (i.e., Adobe Target, Adobe Analytics, Optimizely, Amplitude)

• QA tests tracking, tags, and feature flags prior to test launch and once test is live in production to capture the test data

(70%) Analytics & Reporting:

• Analyzing testing results based on statistical methods to provide statistically significant, sound recommendations to push winning solutions into production for all traffic

• Determine test result statistical significance, level of confidence, when to stop and conclude testing, recommend next steps, and discern key takeaways and learnings

• Define new opportunities to test using data-driven observations and analysis

• Documenting test results/implications, and using this knowledge to iterate upon test concepts and further optimize the customer experience

• Measure and analyze of site tests, ensuring appropriate measurement of performance and impact

• Establish reporting templates for test results that summarize the hypothesis, results and identify opportunities to improve performance

• Use segmentation and click stream data to build tests to personalize and optimize each customer’s online experience while performing customer analysis and segmentation

• Drive input on personalization experiences supported by site data

• Communicate and document test results and produce a weekly product health metrics report to a wide audience at various levels of product experience and business groups.

About You 

• BS or MS degree in technical/analytics field (economics, Mathematics, Statistics, Data Analytics, Data Science, Computer Science, Engineering

• 3-5 years’ experience in a consumer facing digital role in analytics and reporting

• Hands on experience working with Big data, ideally clickstream data, in Big data volume sectors such as eCommerce, gaming, social network etc. eCommerce platforms is a plus

• Solid understanding of statistics and ability to calculate basic statics for A/B tests

• Understanding of technical and analytical approaches, including digital A/B and multivariative testing

• Execution minded analyst including strong tool capabilities (ie: R, SQL, Tableau, Excel/Office, Jira, Confluence, and/or similar)

• Experience in Analytics tools preferably in Adobe Target & Adobe Analytics and/or Amplitude, Optimizely, or similar A/B Testing platform tool

• Excellent written and verbal communication skills; strong interpersonal skills; ability to influence all levels of management.

• Ability to manage and prioritize multiple tasks and adapt to a changing and fast-paced environment

• Collaborative, team player

• Passionate about eCommerce, approachable, thoughtful, respectful to teams and customers