Machine Learning Engineer, AI Evaluation Lab

Key Skills

Aartificial intelligence, Algorithms, Continuous Integration & Continuous Delivery - CI/CD, Deep Learning, Github, HIPAA, HuggingFace, JAX framework, Machine learning techniques, MLOps tools, Python Programming, SQL

Job Description

Stanford University is seeking a Machine Learning Engineer to perform advanced technical research for the AI Evaluation Lab. The AI Evaluation Lab evaluates the performance, safety, effectiveness, reliability, and transparency of AI models intended for real-world deployment. It aims to understand performance degradation in imaging AI applications and develop strategies to recognize, measure, and mitigate these issues. 

Reporting to the Executive Director of the 3D and Quantitative Imaging (3DQ) Lab and with guidance and dotted line reporting to senior faculty leaders, the individual will build end-to-end data pipelines and infrastructure for ML models used in Stanford Radiology. They will build robust and modular software engineering infrastructures for trained and inference of ML models that can be used for a variety of downstream applications and will use their knowledge to make recommendations and design decisions for languages, tools, and platforms used in software and data projects. 

Duties include:

•  Support complex scientific and research programs related to area of specialization; analyze data, monitor and oversee experimental process, and design and develop prototypes, specialized equipment, and/or systems. 
• Collaborate with scientists, engineers, or senior administrative officers to oversee complex non-routine analyses, select optimum solutions, and perform corrective modifications to equipment and system designs. 
• Carry out all activities, including troubleshooting and resolving routine problems for scientist or engineers, independently. 
• Collaborate with senior engineers and scientists to design and develop special purpose equipment and/or systems. 
• Participate in the planning, design, and implementation of scientific or engineering initiatives, and work toward project objective. 
• Oversee a laboratory space or unit, and supervise the work of technicians and other staff associated with the group. 
• Serve as a resource in review of research proposals and research capabilities, and make recommendations. 
• Establish, communicate, and enforce compliance with health and safety policies and procedures. 
• Develop training manuals and safety guidelines, and train new instrumentation users, researchers, and/or technical staff. 
• Perform supervisory duties, including overseeing the work of technicians and other staff associated with the group/project, supervising the regular installation, maintenance, and operation of complex scientific or engineering projects, and training technicians, operators, and others working in particular scientific or engineering function area.

Other duties may also be assigned

DESIRED QUALIFICATIONS:

• Ability to install, configure, and implement machine learning algorithms in modern training platforms (such as PyTorch, JAX) and inference platforms (such as Hugging Face, gradio, streamlit).
• Experience with cloud deployment strategies and CI/CD. 
• Experience overseeing, developing or implementing machine learning operations (MLOps) processes. Experience working with healthcare data. 
• Experience in building software and data infrastructure for analytics team, including ability to write Python and SQL for processing large datasets.
• Research and prototype state-of-the-art computer vision and deep learning methods to develop solutions that meets product requirements. 
• Design algorithm evaluation frameworks, schedule, and report algorithm performance on a regular basis. 
• Work in shared code environments such as GitHub and collaborate with other developers and be responsive to GitHub issues and pull requests. 
• Lead code reviews for projects/systems as an independent reviewer applying design principles, coding standards and best practices
• Experience working in a HIPAA regulated environment.
• Experience with publications in deep learning for medical imaging in healthcare journals or ML conferences a plus.

EDUCATION & EXPERIENCE (REQUIRED):

Bachelor’s degree in engineering, science, or related field and three years of relevant experience; or a combination of education and relevant experience.

KNOWLEDGE, SKILLS AND ABILITIES (REQUIRED):

• Demonstrated knowledge and skills of advanced scientific or engineering principles and practices. 
• Demonstrated experience applying complex scientific and engineering principles and performing special technical services involving both development and performance. 
• In-depth experience with software applications, systems, or programs relevant for the job. 
• Ability to independently oversee and manage instrumentation or system installation. 
• Ability to collaborate with senior engineering and scientific staff to design and develop special purpose equipment and/or systems. 
• Experience overseeing the plan, design, and implementation of major scientific or engineering initiatives and ensuring project objective are met. 
• Demonstrated ability to review research proposals, evaluate research capabilities, and make recommendations. 
• Demonstrated ability to establish, communicate, and enforce compliance with health and safety policies and procedures. 
• Experience overseeing a laboratory space or unit and supervising the work of technicians and other staff associated with the group.
• Demonstrated ability to effectively supervise and train a diverse work staff.

PHYSICAL REQUIREMENTS:

• Frequently grasp lightly/fine manipulation, perform desk-based computer tasks, lift/carry/push/pull objects that weigh up to 10 pounds.
• Occasionally stand/walk, sit, twist/bend/stoop/squat, grasp forcefully.
• Rarely kneel/crawl, climb (ladders, scaffolds, or other), reach/work above shoulders, use a telephone, writing by hand, sort/file paperwork or parts, operate foot and/or hand controls, lift/carry/push/pull objects that weigh >40 pounds.

Consistent with its obligations under the law, the University will provide reasonable accommodation to any employee with a disability who requires accommodation to perform the essential functions of his or her job.

WORKING CONDITIONS:

• May be exposed to high voltage electricity, radiation or electromagnetic fields, lasers, noise > 80dB TWA, Allergens/Biohazards/Chemicals /Asbestos, confined spaces, working at heights ?10 feet, temperature extremes, heavy metals, unusual work hours or routine overtime and/or inclement weather. 
• May require travel.

WORK STANDARDS: 

• Interpersonal Skills: Demonstrates the ability to work well with Stanford colleagues and clients and with external organizations.
• Promote Culture of Safety: Demonstrates commitment to personal responsibility and value for safety; communicates safety concerns; uses and promotes safe behaviors based on training and lessons learned.
• Subject to and expected to comply with all applicable University policies and procedures, including but not limited to the personnel policies and other policies found in the University's Administrative Guide, http://adminguide.stanford.edu.

The expected pay range for this position is $109,000 to $129,000 per annum.

Stanford University provides pay ranges representing its good faith estimate of what the university reasonably expects to pay for a position. The pay offered to a selected candidate will be determined based on factors such as (but not limited to) the scope and responsibilities of the position, the qualifications of the selected candidate, departmental budget availability, internal equity, geographic location and external market pay for comparable jobs.