Research Scientist Intern, Sensor Fusion/ML (PhD)

Key Skills

Computer Vision (CV), Geometry, Machine learning techniques, Numerical Computing, Object detection, Optimization, Python Programming, PyTorch, Statistical modeling, Text segmentation

Job Description

Meta Reality Labs (RL) Redmond is looking for exceptional interns to help create the next generation of virtual and augmented reality. Our research focuses on true solutions to complex tracking problems, delivering novel technologies and algorithms fundamental to future VR and AR systems. You’ll have the unique opportunity to work with industry-leading researchers and engineers who are defining the future of VR.

In particular, we welcome candidates who strive for a deep understanding of complex systems, and a strong desire to push beyond state-of-the-art. We also welcome students with a passion for working in a strongly collaborative team that spans research across algorithms, hardware design and systems prototyping.

Our internships are twelve (12) to sixteen (16) to twenty four (24) weeks long and we have various start dates throughout the year.

Our research topics include (but are not limited to) the following areas:

• Sensor Fusion, Computer vision and machine learning, with an emphasis on eyes and/or faces
• Geometric modeling and non-linear optimization for compact representation of high-dimensional functions.
• Photometric reconstruction of eyes and faces using advanced capture and rendering techniques
• Statistical modeling of oculomotor systems and sensor fusion algorithms
• Eye movement prediction and perceptual control loops
• Scene and Video analysis, Multi-modal sensor and interaction fusion, Egomotion data-analysis, Object Detection/Segmentation
• Disentangled-latent representation learning for image synthesis in 2D and 3D
• Temporal sequence learning for gaze estimation and prediction
• Statistical machine learning for uncertainty analysis and prediction

Research Scientist Intern, Sensor Fusion/ML (PhD) Responsibilities

• Implement published models for appearance based gaze estimation and tailor to suit applications in AR/VR domain
• Collaborate with researchers and engineers across diverse disciplines
• Work closely with a proactive, highly motivated engineering team to bring future generation object tracking and sensor fusion solutions to eye tracking team at Meta
• Statistical analysis and evaluation of regression models for gaze estimation
• Research, design, develop and evaluate advanced sensor fusion and state estimation algorithms for an object tracking pipeline including but not limited to multi-object tracking, segmentation, object detection and classification, and sensor fusion
• Develop state of the art object tracking and sensor fusion algorithms for multiple sensors
• Tasks include developing, evaluating, benchmarking and deployment into real-time pipelines
• Maintain and improve our existing in-house algorithms and models, including continuous evaluation, gap analysis, re-training and fine tuning
• Develop evaluation scripts to process large data and accurately measure algorithmic and end to end performance

Minimum Qualifications

• Currently has, or is in the process of obtaining, a PhD degree in the field of Computer Science, Applied Math, Physics or a related STEM field
• High levels of creativity and quick problem solving capabilities
• Interpersonal skills: cross-group and cross-culture collaboration
• Must obtain work authorization in country of employment at the time of hire, and maintain ongoing work authorization during employment
• Strong python/pytorch, familiarity with software development best practices, debugging/profiling
• Understanding of statistical multi-object tracking, computer vision, motion state estimation, sensor fusion, and machine learning
• Experience with multiple object tracking and with one or more perception sensors (radar, camera, lidar, ultrasonics, etc.)

Preferred Qualifications

• Demonstrated software engineer experience via an internship, work experience, coding competitions, or widely used contributions in open source repositories (e.g. Github)
• Experience with sensor fusion methods, Pytorch, numerical optimization
• Experience with basic optical systems, with an emphasis on the optics of the human eye
• Experience implementing machine learning algorithms with Python
• Intent to return to degree-program after the completion of the internship
• Proven track record of achieving results as demonstrated in accepted papers at top sensor fusion and/or machine learning related conferences such as ICLR, ICML, NeurIPS, etNeurIPS, ETRA, SIAM Conference on Mathematics of Data Science, International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), etc

About Meta

Meta builds technologies that help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and WhatsApp further empowered billions around the world. Now, Meta is moving beyond 2D screens toward immersive experiences like augmented and virtual reality to help build the next evolution in social technology. People who choose to build their careers by building with us at Meta help shape a future that will take us beyond what digital connection makes possible today—beyond the constraints of screens, the limits of distance, and even the rules of physics.

Meta is committed to providing reasonable support (called accommodations) in our recruiting processes for candidates with disabilities, long term conditions, mental health conditions or sincerely held religious beliefs, or who are neurodivergent or require pregnancy-related support. If you need support, please reach out to accommodations-ext@fb.com.

$7,500/month to $11,333/month + benefits

Please note the national salary range listed in the job posting reflects the new hire salary range across levels and U.S. locations that would be applicable to the position. Final salary will be commensurate with the candidate’s final level and final location. Also, this range represents base salary only and does not include the company bonus, incentive for sales roles, equity, or benefits , if applicable.