AI/ML - Engineering Manager, ML Systems Evaluation Platforms

Key Skills

Apache Hadoop, C Programming, C++, Data science techniques, Java Programming, Machine learning techniques, MapReduce, Python Programming, PyTorch, R Programming, Scala Programming, SQL, TensorFlow

Job Description

Would you like to play a part in ensuring the quality of groundbreaking technology for large-scale systems, natural language, big data, and artificial intelligence? Drive the quality of the Siri user experience and work with the people who built the intelligent assistant that helps millions of people get things done — just by asking? The engineering manager will own experimental-testing tools development, ensuring our quality measurement of Siri and ML products pre-release is predictive of real-life user experience. This role is responsible for developing the approach, tools, and testing for systems integration (client/server and data/server), reliability, and consistency measurements. As Siri evolves, testing methodology and tools must respond to the new complexities: delivering such alignment is a key requirement for this role. An important deliverable of the role is a significant improvement in failure actionability, combined with identifying systemic issues, and with failures prioritization. This is a fast-paced role with high visibility, impact, and helps to build the most advanced systems in the industry.

Key Qualifications

• 5+ years of professional work experience in development.
• 3+ years of experience managing an engineering team.
• The ideal candidate has experience with test infrastructure, quality tools and automation, and system integration testing.
• Self-motivated and dedicated with proven creative and critical thinking capabilities.
• Experience working with large-scale server software features, test planning, and execution.
• Fluent programming in Java (or advanced usage of OO languages such as C++, Python, Obj-C, or Swift).
• Proficient with scripting in Python, Perl, Bash.
• Working experience with different logging and analytics frameworks.
• Experience with statistical analysis applications in testing, ML/AI, or manufacturing processes
• Ability to build test automation frameworks.
• Experience with root cause analysis of multi-tiered, client-server environments.
• Testing server backend services, especially at the API or protocol level.
• Familiarity with testing machine learning systems.
• Strong organizational skills and experience working on large multi-functional teams.
• Thrives on working in a fast-paced environment with constantly evolving priorities.
• Ability to lead large teams, develop a long-term vision, and execute strategies at scale.

Description

Owns the quality engineering experimental testing roadmap for Siri features. Owns the development of experimental testing tools, with a focus on statistical testing and E2E view. Develops and applies various statistical approaches to building E2E test sets. Explores production data to discover gaps in testing. Owns failure triage automation; focuses on significantly improving failure actionability. Owns systems integration testing, complete with test plan and tools development (client/server and data/server). Develops success criteria for QE tools testing, and oversees such testing execution and feedback to engineering partners. Thinks tirelessly of strategic and creative ways to evaluate and improve Siri user experience, and drives the team to work towards this shared goal. Builds the team on the foundation of excellence, professional growth, inclusivity, and respect.

Education & Experience

BS/MS in Computer Science or related field.