Product Operations - Product Operations Manager, AI Operations

Meta•Austin, TX

1d•$123,000 - $179,000

About The Position

Meta is seeking a Product Operations Manager to join the Foundations Product Operations team. Our mission is to be the experts on quality to improve how product teams resolve user issues through AI. Those who join our team are passionate about solving user issues and are advocates for the Meta community. We need influencers who can align our engineering customers and cross-functional partners to ensure the best possible experience for our platforms. If you like working with data and helping users, Product Operations is for you. We are looking for an individual contributor with data and AI expertise who can partner with cross functional teams in complex, multi-platform user response quality efforts to build AI agents (end-to-end model lifecycle), manage the evolving ecosystem of LLM models and establish a scalable model EVALs program. This role will help ensure that Product Operations has scalable, measurable and consistent quality programs (across team-wide processes, metrics and tooling) to effectively drive production quality, development quality and product reliability by building, managing and maintaining a healthy LLM ecosystem that evolves with the growing needs of our product partners and users. Many programs need to be developed or implemented with new approaches such as our aggregated customer reporting, quality and efficiency programs with our triage specialists, driving tool and process improvements, as well as evolving our metrics and goaling operations. You’ll collaborate closely with Product, Engineering, Research, and Operations to define signals, run user-based experiments, and elevate our quality metrics across internal and external use cases.

Requirements

Bachelor's degree in a directly related field, or equivalent practical experience
8+ years experience in product operations, ML evaluation, user research, or similar roles
Proven track record managing large-scale, complex projects and programs that impact entire Operations and Product organizations
Analytical skills, proficiency with data tools (SQL, Python, experimentation platforms), using data to tell a story and influence product direction
Demonstrated experience building data pipelines, reports, dashboards (Unidash etc.), and visualizations to effectively communicate findings to executive technical stakeholders to influence decision-making
Proactive problem solver with experience breaking down ambiguous issues into component parts to develop solutions
Executive communication skills, particularly written communications and demonstrated experience engaging with leaders
Stakeholder management— influencing cross-functional teams with qualitative and quantitative insights

Nice To Haves

Experience with LLM Model development and management of LLM/ Agent Ecosystems
Knowledge and understanding of EVALs and AI Agent performance management
Experience working with global/remote teams
Previous leadership and personnel management experience

Responsibilities

Establish and lead ongoing programs and systems in a fast paced, innovation-driven environment to build our critical AI model foundation, scaling best practices while also ensuring proactive and high quality model performance
Collect, analyze, and leverage data from various sources to identify trends, patterns, and insights that can inform business decisions
Leverage SQL and data visualization tools to monitor product metrics, understand AI trends, and support end-to-end agent optimization
Leverage AI and automation models, prompts, and insights to optimize workflow efficiency and improve quality of our output, insights, and deliverables
Define and lead program execution strategy for multiple areas of products and the platform, including kickstarting 0 to 1 efforts, accelerating execution, and improving quality/outcomes for product objectives via programmatic solutions
Foster partnerships with Product, Engineering, and Product Operations leaders to align goals, streamline communication, and support business priorities as well as implement product, process, and tool improvements
Mentor team members, providing continual support to unblock and enable them to complete their respective projects
Design, build, and improve processes and systems that contribute to scalable AI solutions