AIML - Senior ML Product Evaluation Engineer

Key Skills

Aartificial intelligence, Continuous Integration & Continuous Delivery - CI/CD, Data science techniques, Design, Github, Jenkins, Leadership Skill, Machine learning techniques

Job Description

As a Senior ML Product Evaluation Engineer, you’ll play a key role in shaping AI-powered products to deliver an exceptional user experience. This role requires a solid understanding of both machine learning and software engineering. You’ll lead the evaluation of ML models in large-scale systems, designing strategies that assess both feature stability and real-world model effectiveness. Collaborating with software developers, data scientists, and product managers, you will play a key role in delivering high-quality products used by millions worldwide. If you're passionate about functional testing, ML model evaluation, and seamless integration across complex platforms, this role is for you!

In this role, you will:

• Lead the design and execution of test plans for features across various platforms.
• Test and evaluate the ML models powering Siri for accuracy, performance, and stability.
• Create datasets and conduct model performance evaluations to ensure ML models meet required standards.
• Debug complex issues by analyzing logs and collaborating with developers to resolve root causes efficiently.
• Provide detailed test reports, highlight risks, and ensure issues are addressed before product release.
• Collaborate with data scientists and ML engineers to validate model deployment and performance in production environments.

Minimum Qualifications

• 7+ years of experience with ML model testing and evaluation in production environments.
• 7+ years of experience in working with distributed systems that include multiple sub-systems and orchestration components.
• Strong experience in creating datasets for model evaluation and conducting performance benchmarks.
• Advanced debugging skills, including log stream analysis and issue reproduction.
• Familiarity with ML model testing tools like TensorFlow, PyTorch, or MLflow.
• Proactive and creative mindset with a can-do attitude and strong focus on delivering high-quality results.

Preferred Qualifications

• Experience validating the performance and scalability of machine learning models in a production setting.
• Ability to lead and influence testing initiatives in fast-paced, dynamic environments.
• Knowledge of Swift, XCTest or equivalent tools is a plus.
• Proficiency in configuring and maintaining CI/CD pipelines using tools such as GitHub, TeamCity, Jenkins, or similar platforms.
• Master’s degree in Machine Learning, Data Science, or related field. PhD in Machine Learning or Artificial Intelligence is preferred