- Professional
- Optionales Büro in Mountain View
Minimum qualifications:
- Master's degree in Statistics, Data Science, Mathematics, Physics, Economics, Operations Research, Engineering, or a related quantitative field.
- 3 years of experience using analytics to solve product or business problems, coding (e.g., Python, R, SQL), querying databases or statistical analysis, or a PhD degree.
Preferred qualifications:
- Experience in developing applications or agents with Large Language Models (LLMs).
- Ability to prioritize projects and communicate effectively with stakeholders across engineering, product, and UX research teams.
About the job
The Core Ranking Data Science team generates product insights, including those derived from direct user feedback, and develops and supports evaluation methodology and metrics to guide and assess Search Quality models and the overall product. Our mission is to drive product excellence, and user and ecosystem growth for Search with data insights. We work with engineers, researchers, and product managers within and outside of Search to bring helpful and trustworthy content to Search users.
Among other things, we work on developing quality metrics and evaluation methodologies, incorporating human, AI (LLM-based raters), and direct user feedback signals. We leverage user feedback and other data sources to uncover strategic insights, quantify headroom, and influence the Search Quality roadmap. Our team drives innovation in Search Quality experimentation and in deepening our understanding of the user experience. Additionally, we build and scale AI agents to automate and enhance evaluation, insights generation, and data science workflows.
The US base salary range for this full-time position is $141,000-$202,000 + bonus + equity + benefits. Our salary ranges are determined by role, level, and location. Within the range, individual pay is determined by work location and additional factors, including job-related skills, experience, and relevant education or training. Your recruiter can share more about the specific salary range for your preferred location during the hiring process.Please note that the compensation details listed in US role postings reflect the base salary only, and do not include bonus, equity, or benefits. Learn more about benefits at Google.
Responsibilities
- Design and develop AI agents to automate and scale data science workflows, including A/B test analysis, evaluation, insights generation, and metric development.
- Design and implement metrics to evaluate AI agent performance, quality, and reliability. Analyze in-product experiments (e.g., side-by-sides) to assess the user experience of AI-powered features.
- Research, adapt, and apply techniques in LLMs, and Gen AI agents to create innovative solutions for Search Data Science.
- Collaborate cross-functionally with engineering, research, and product teams to define agent requirements, plan and execute in-product side-by-side experiments, and drive data-informed product improvements.
- Communicate findings, agent capabilities, and experiment outcomes to stakeholders.
