Generative AI Analyst, Trust and Safety Operations

Key Skills

Aartificial intelligence, Analytical and Problem solving, Business Intelligence, Generative AI, Large Language Models - LLMs, Python Programming, SQL

Job Description

Minimum qualifications:

• Bachelor's degree in Computer Science, Engineering, Statistics, Mathematics, a related discipline, or equivalent practical experience.
• 4 years of experience working with analytical tools or data analysis (e.g., SQL, Python, business Intelligence/dashboards).

Preferred qualifications:

• Experience working with spam, fraud, or malware.
• Experience working on product policy analysis and identifying policy risks.
• Experience using statistical analysis and hypothesis testing.
• Experience analyzing machine learning models performance or working on Large Language Models (LLM).
• Excellent influencing skills, with the ability to work in a dynamic environment.
• Excellent problem-solving and critical thinking skills with attention to detail in an ever-changing environment.

About the job

Trust & Safety team members are tasked with identifying and taking on the biggest problems that challenge the safety and integrity of our products. They use technical know-how, excellent problem-solving skills, user insights, and proactive communication to protect users and our partners from abuse across Google products like Search, Maps, Gmail, and Google Ads. On this team, you're a big-picture thinker and strategic team-player with a passion for doing what’s right. You work globally and cross-functionally with Google engineers and product managers to identify and fight abuse and fraud cases at Google speed - with urgency. And you take pride in knowing that every day you are working hard to promote trust in Google and ensuring the highest levels of user safety.

The Trust and Safety team is working with technologies in order to keep users safe and improve the quality of Google's products. As part of the team, you will develop and manage cross-functional relationships with Engineering, Product, Legal, and Operations teams in efforts to improve processes for identifying and resolving policy and quality issues on Generative AI products.

In this role, you will be responsible for the execution of projects with a diverse group of stakeholders. You will analyze market trends, set forward-looking strategy, and implement scaled solutions to solve user safety and quality issues, and ensure a positive user experience across Generative AI products.

At Google we work hard to earn our users’ trust every day. Trust & Safety is Google’s team of abuse fighting and user trust experts working daily to make the internet a safer place. We partner with teams across Google to deliver bold solutions in abuse areas such as malware, spam and account hijacking. A diverse team of Analysts, Policy Specialists, Engineers, and Program Managers, we work to reduce risk and fight abuse across all of Google’s products, protecting our users, advertisers, and publishers across the globe in over 40 languages.

Responsibilities

• Establish metrics to benchmark Generative AI issues and quantify improvements, develop datasets and run evaluations for engineers to evaluate and improve Search, Labs, and Assistant products, and perform data analyses to derive insights that help identify and fight abuse.
• Use technical judgment to drive project delivery, challenge proposals, and identify risks. Manage multiple projects and scope against crashed timelines.
• Improve tools through data analysis, technical expertise, and presentations, as well as enhance operational workflows via process improvements and identification of automation opportunities.
• Analyze and debug escalations, identify trends that can indicate product risks, and perform on-call responsibilities on a rotating basis, including weekend coverage.
• Review graphic, controversial, and offensive/sensitive content as needed.