Staff Product Data Scientist, Workspace

Google • New York, NY

1d • $183,000 - $271,000

About The Position

In this role, you will dive deep, dissecting model performance from foundational layers to product specific fine-tuning. You will balance how to blend techniques like autoratings, classic human eval, and reveal preference modeling to illuminate where we need to do better, and where we are doing well. Your findings will drive cross-organizational engineering efforts, shaping the very core of our AI-powered future. You will have the opportunity to work with technology and contribute to the development of a product that has the potential to transform the way people work.

Requirements

• Bachelor's degree in Statistics, Mathematics, Data Science, Engineering, Physics, Economics, or a related quantitative field.
• 10 years of experience using analytics to solve product or business problems, performing statistical analysis, and coding (e.g., Python, R, SQL) or 8 years work experience with a Master's degree.

Nice To Haves

• Master's degree in Statistics, Mathematics, Data Science, Engineering, Physics, Economics, or a related quantitative field.
• 12 years of experience using analytics to solve product or business problems, performing statistical analysis, and coding (e.g., Python, R, SQL).

Responsibilities

• Conduct evaluations of our model performance, focusing on productivity-related tasks and workflows across modalities (text, video, images, speech).
• Develop and implement evaluation frameworks and metrics to assess the model's strengths and weaknesses.
• Analyze evaluation data and generate reports to communicate findings to leadership.
• Collaborate closely with research and engineering teams to identify areas for improvement and contribute to the development of new evaluation methodologies.
• Stay up-to-date with the latest advancements in LLM technology and evaluation techniques, and be conversant in (and understand the shortcomings of) industry benchmarks.

Benefits

• Health, dental, vision, life, disability insurance
• Retirement Benefits: 401(k) with company match
• Paid Time Off: 20 days of vacation per year, accruing at a rate of 6.15 hours per pay period for the first five years of employment
• Sick Time: 40 hours/year (statutory, where applicable); 5 days/event (discretionary)
• Maternity Leave (Short-Term Disability + Baby Bonding): 28-30 weeks
• Baby Bonding Leave: 18 weeks
• Holidays: 13 paid days per year

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level