Senior/Staff Machine Learning Engineer - Foundation Models

Key Skills

AI Robotic systems, Autonomous vehicles, C++, Deep Learning, Foundation Models, JAX framework, Large Language Models - LLMs, Machine learning techniques, Python Programming, PyTorch, TensorFlow

Job Description

The Prediction & Behavior ML team is responsible for developing machine-learned models that understand the full scene around our vehicle and forecast the behavior for other agents, our own vehicle’s actions, and for offline applications.

To solve these problems we develop deep learning algorithms that can learn behaviors from data and apply them on-vehicle to influence our vehicle’s driving behavior and offline to provide learned models to autonomy simulation and validation. Given the tight integration of behavior forecasting and motion planning, our team necessarily works very closely with the Planner team in the advancement of our overall vehicle behavior. The Prediction & Behavior ML team also works closely with our Perception, Simulation, and Systems Engineering teams on many cross-team initiatives.

The Behavior Foundation Model team is responsible for building GPT (Generative Pre-Trained) for autonomous driving and ecosystems for prediction and planning for autonomous driving and offline applications. We're looking for experts in this field to come work with us in building these models and applying them to downstream tasks across the company.

Responsibilities

• You will build the foundation models for on-vehicle and offline applications
• You will push the boundaries of machine learning in the autonomous driving industry by developing state-of-the-art techniques in self-supervised learning, semi-supervised learning, multi-task learning, and generative models
• You will leverage our large-scale machine learning infrastructure to discover new solutions
• You will develop new algorithms to model the future behavior of all other agents in the world
• You will develop new algorithms to model the future behavior of our own vehicle’s future actions, both in predicting our driving trajectories and estimating their quality in relation to our goals of safety, progress, and comfort
• You will develop new algorithms to apply generative deep learning to simulation to improve the realism of our offline validation systems
• You will engineer software that runs on-vehicle to efficiently execute our algorithms in real time
• You will develop metrics and tools to analyze errors and understand improvements of our systems
• You will collaborate with engineers on Perception, Prediction, Planning, Simulation, and Systems Engineering to solve the overall Autonomous Driving problem in complex urban environments

Qualifications

• BS, MS, or PhD degree in Computer Science or a related field
• Fluency with PyTorch, TensorFlow, or JAX
• Experience with training and deploying Deep Learning models
• Experience with production Machine Learning pipelines: dataset creation, training frameworks, metrics pipelines
• Fluency in Python and a basic understanding of C++
• Extensive experience with programming and algorithm design
• Strong mathematics skills

Bonus Qualifications

• Experience in training of large and complex models
• Experience with LLMs
• Conference or Journal publications in Machine Learning or Robotics
• Prior experience with Prediction and/or Autonomous Vehicles or Robotics in general

Compensation

There are three major components to compensation for this position: salary, Amazon Restricted Stock Units (RSUs), and Zoox Stock Appreciation Rights. The salary will range from $221,000 to $319,000. A sign-on bonus may be part of a compensation package. Compensation will vary based on geographic location, job-related knowledge, skills, and experience. 

Zoox also offers a comprehensive package of benefits including paid time off (e.g. sick leave, vacation, bereavement), unpaid time off, Zoox Stock Appreciation Rights, Amazon RSUs, health insurance, long-term care insurance, long-term and short-term disability insurance, and life insurance.