AI Safety & Robustness Analysis Manager - System Intelligent and Machine Learning - ISE

Key Skills

Computer Vision (CV), Deep Learning, Large Language Models - LLMs, Machine learning techniques, Natural Language Processing (NLP), PyTorch

Job Description

In this position, you will manage a team of people passionate about leading RA operations for key future facing Apple features with focus on ensuring safety and robustness for generative models. Apple’s dedication to deliver incredible experiences to a global and diverse set of users, in full respect of their privacy, has led to the development of a dedicated Robustness Analysis function. With the generative experience, creating a safe and robust platform is vital to our mission. Team’s responsibilities include monitoring ML model performance on relevant axes, and surfacing, measuring and mitigating ML failure modes, in order to improve overall user experience and reduce risks, with specific attention given to safety, inclusion and fairness.

THE TEAM’S RESPONSIBILITIES INCLUDE:

• research and develop approaches to mitigate harmful and risk behaviors in generative models

• define product-centered axes of analysis relevant to target feature, in collaboration with model DRI and feature DRI

• develop processes (models, tools and data) to identify other potential biases or failure modes

• implement automated pipelines based on advanced ML technology and humans/models in the loop to create test sets covering the various axes of investigation

• report progress and issues found in technical and sponsor meetings

• suggest mitigation options (data and/or model) and lead mitigation experiments, when issues are found

• become key contact within our organization, in company wide efforts related to safety, fairness and inclusion, robustness analysis, interpretability

Key Qualifications

• 2+ years of experience as a machine learning manager, or 7+ years of professional machine learning experience with demonstrated technical leadership
• Capacity to build a team and establish innovative and agile processes that push for a high level of service and for scalability
• Strong ML fundamentals and hands-on experience in training ML models (NLP or Computer Vision), familiarity with ML toolkits, e.g., PyTorch
• Proven experience in accessing and addressing potential risks and ensuring the safety and fairness in generative models. Prior experience in LLM Safety is desired
• Strong experience in quantitative methods, data analysis and machine learning, leading to a deep understanding of the challenges associated to building ML datasets and machine learning models (potential biases, potential failure modes)
• Capacity to operate at the intersection of ethics, product experience and machine learning: translate product needs (e.g. fairness & safety) into analytical requirements; design and lead experiments to answer feature level questions

Education & Experience

M.S. or PhD in Computer Science, Data Science, Mathematics, Physics, or a related field; or equivalent practical experience

Pay & Benefits

At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $183,400 and $275,600, and your base pay will depend on your skills, qualifications, experience, and location.

Apple employees also have the opportunity to become an Apple shareholder through participation in Apple’s discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple’s Employee Stock Purchase Plan. You’ll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses — including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits.

Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.