Technial Delivery Manager—AB Testing

Papa Johns

About The Position

What’s Unique About You Is What Makes Us Better! Diversity is our strength and competitive advantage. Bring your flavor to the Papa John's team today! Job Summary The Technical Delivery Manager, Digital Technology focused on Experimentation & Ecommerce Optimization leads the end-to‑-‑end delivery of AB tests, multivariate experiments, and product/menu/offers/campaigns variation initiatives that drive measurable improvements in conversion, order value, and customer engagement across ecommerce channels. This role manages the technical execution of experimentation roadmaps and partners closely with Data Science, Finance, Marketing, Operations, and Consumer Insights to ensure tests are designed rigorously, executed reliably, and translated into business impact. The ideal candidate combines technical fluency, structured delivery leadership, and the ability to turn business hypotheses into statistically sound experimentation outcomes.

Requirements

Bachelor’s degree in Computer Science, Information Systems, Engineering or related field.
5+ years of experience in technical delivery management, program management, or experimentation delivery in digital or ecommerce environments.
Experience coordinating across engineering, data science, analytics, and business teams.
Familiarity with AB testing platforms, feature flagging tools, and ecommerce data flows.
Proficiency in Python and SQL, with strong exploratory data analysis skills and experience working with real-world‑ datasets.
Foundational knowledge of data pipelines, APIs, and system design principles that support ML integration into production environments.
Analytical mindset with the ability to frame success metrics, evaluate model performance, and interpret results beyond basic accuracy.
Strong written and verbal communication skills, with the ability to explain complex technical ideas to non‑technical stakeholders.

Nice To Haves

Advanced academic background in Data Science or ML
Understanding of core ML concepts and the ability to translate theoretical models into working prototypes.
Familiarity with modern AI architectures, including LLM-‑based workflows, multiagent systems, and tool‑ integration‑ patterns.
Agile, Scrum, or PMP certifications preferred.
Experience in QSR, retail, or high-velocity‑ ecommerce environments is a plus.

Responsibilities

Manage the delivery of AB tests, feature flags, and menu/product/offers/campaigns and digital customers’ experiences variation experiments across multiple ecommerce domains.
Translate business hypotheses into clear technical requirements, test plans, and delivery timelines.
Coordinate execution across engineering, data science, analytics, QA, and external vendors to ensure tests launch reliably and on schedule.
Maintain a predictable experimentation pipeline with proper prioritization, sequencing, and resource allocation.
Collaborate with Data Science to define test design, sample sizing, segmentation, and statistical methodologies.
Ensure data pipelines, instrumentation, and event tracking meet the standards required for accurate measurement.
Partner with Data Science and Consumer Insights to validate experiment readiness, interpret results, and translate findings into actionable recommendations.
Support the development of experimentation frameworks, governance models, and measurement standards.
Establish delivery standards for experimentation, including data integrity, governance, and platform scalability.
Drive continuous improvement in experimentation tooling, automation, and operational efficiency.
Ensure all tests adhere to statistical rigor, data quality requirements, and business KPIs.
Serve as the primary delivery interface for Finance, Marketing, Operations, Consumer Insights, and Data Science.
Align experimentation outcomes with business KPIs such as conversion rate, order ticket, attach rate, and customer engagement.
Facilitate alignment on test hypotheses, success metrics, operational impacts, and rollout decisions.
Communicate progress, risks, and results to senior stakeholders with clarity and business context.
Forecast delivery costs, resource needs, and expected ROI for experimentation initiatives.
Ensure efficient use of engineering, data science, and vendor capacity while maximizing business value.
Identify delivery risks early and make informed trade‑offs across speed, quality, and business impact.
Escalate blockers proactively and drive resolution across teams.
Ensure all tests meet data, privacy, and operational compliance requirements.
Mentor delivery managers and senior ICs, fostering strong execution discipline and strategic thinking.
Promote a culture of experimentation, collaboration, and continuous learning across teams.