Senior Data Scientist, Research, Generative AI Factuality

Key Skills

Data science techniques, Planning and organization, Python Programming, R Programming, SQL

Job Description

Minimum qualifications:

• Master's degree in Statistics, Data Science, Mathematics, Physics, Economics, Operations Research, Engineering, a related quantitative field, or equivalent practical experience.
• 5 years of work experience using analytics to solve product or business problems, coding (e.g., Python, R, SQL), querying databases or statistical analysis, or 3 years of work experience with a PhD degree.

Preferred qualifications:

• 8 years of work experience using analytics to solve product or business problems, coding (e.g., Python, R, SQL), querying databases or statistical analysis, or 6 years of work experience with a PhD degree.
• Experience with LLM evaluation, including autoraters and log analysis.

About the job

In this role, you will help evaluate and improve the quality of tools and products that apply AI to improve access to accurate and helpful information.

You will be responsible for Gemini's factuality and quality on information-seeking prompts. You will ensure that models are as accurate as possible without compromising any other response quality aspect, and that response structure and organization are helpful. You will also be responsible for factuality evaluations in Gemini, including defining factuality metrics, eval templates, and building high-quality autoraters. You will partner with various Gemini verticals: Factuality, Quality, Multi-Turn, Multilinguality, and Model Behavior.

The US base salary range for this full-time position is $166,000-$244,000 + bonus + equity + benefits. Our salary ranges are determined by role, level, and location. Within the range, individual pay is determined by work location and additional factors, including job-related skills, experience, and relevant education or training. Your recruiter can share more about the specific salary range for your preferred location during the hiring process.

Please note that the compensation details listed in US role postings reflect the base salary only, and do not include bonus, equity, or benefits. Learn more about benefits at Google.

Responsibilities

• Collaborate with stakeholders in cross-projects and team settings to identify and clarify business or product questions to answer, and provide feedback to translate and refine business questions into tractable analysis, evaluation metrics, or mathematical models.
• Use custom data infrastructure or existing data models as appropriate, and design and evaluate models to mathematically express and solve defined problems with limited precedent.
• Gather information, business goals, priorities, and organizational context around the questions to answer, as well as the existing and upcoming data infrastructure.
• Own the process of gathering, extracting, and compiling data across sources via relevant tools (e.g., SQL, R, Python). Format, re-structure, or validate data to ensure quality, and review the dataset to ensure it is ready for analysis.