Applied Machine Learning Research Engineer

Key Skills

AWS, Docker, Flask, Google Cloud Platform (GCP), ONNX, Pipeline automation, Python Programming, PyTorch, REST API, YOLO architectures

Job Description

About the Role:

Roboflow is hiring our first research engineer on the machine learning team to drive research initiatives that improve the growing suite of machine learning tools that underpin the Roboflow application. Roboflow is actively used by thousands of engineers and it is now imperative for us to keep our machine learning tooling at the cutting edge and to push the state of the art forward in the ways that we are uniquely positioned, given the breadth of computer vision applications that our product is capturing.

This role will involve adapting and extending open-source machine learning technologies into usable and scalable software. Wide-ranging curiosity and enthusiasm for diving into abstract problems, coming up with good solutions, and seeing them through to completion are essential responsibilities.

Our core belief is that computer vision is a foundational technology that is going to transform nearly every industry. This is an opportunity to shape how millions of developers will experience and use it for the first time. Your contribution will have a massive impact.

What We Need from You

On the machine learning team, we primarily work on building and maintaining technology within Roboflow’straining, search, and deployment services, but from time to time we're also helping deliver on enterprise contracts, and coding awesome open source projects and sample projects. We currently support the fine-tuning and deployment of object detection, classification, instance segmentation, and semantic segmentation models. In the beginning, you will be executing on research initiatives that we have embarked on with partner organizations - centered around the Roboflow 100 object detection benchmark. As we discover results within these initial projects, the role will branch into your own curiosities in the intersection of computer vision research, AI research, and the Roboflow application. If you need a rigid list of tasks spelled out in a multi-month roadmap, this role probably will not be a good fit.

We’re especially keen to add someone to our team who has deep experience and passion in the field and will help us live on the frontier of computer vision technology.

Technology

Our goal is to build the world's best computer vision paradigm and corresponding code infrastructure to back our customer’s applications. This means we handle a lot of challenging complexities like seamlessly ingesting dozens of data formats, processing millions of images per day, and deploying auto-scaling machine learning infrastructure that can handle our customers' most demanding training and deployment needs.

Our core app sits atop Firebase with assistance from auto-scaling groups of Docker containers (for jobs like archiving datasets and training models). We also heavily lean on serverless infrastructure so we can gracefully deal with bursty traffic involved in manipulating datasets that can range anywhere from one hundred to one million images.

Our machine learning infrastructure runs in AWS, with a few deployments spanning into GCP. We train and deploy various state-of-the-art models in a variety of machine-learning frameworks. All of our machine learning applications are closely integrated with the core Roboflow web application.

We also maintain a library of Colab notebooks our customers can use to train common computer vision models, a directory of public datasets, and aweb of format specifications. We see building and supporting mini-projects like these that are helpful to the community at large as part of our role in democratizing computer vision.

Skills - you should be familiar with many of these concepts and technologies and have built projects with some of them:

• Machine Learning Research: PyTorch, ONNX, TensorRT, TFjs, YOLO, Transformers, COCO

• Backend + Cloud: node, Docker, python, Flask, pip, REST, AWS, GCP

We anticipate the role being focused 50% on machine learning research and 50% on implementing results of your research into the Roboflowapplication.You certainly don't need to be an expert in all of these areas; but should be excited to learn new skill sets as you need them. We also hope you'll bring some new knowledge and experiences you can share to help level-up the rest of the team. Your opinions on which research ideas we tackle and how we integrate them will be highly valued.

Example Projects

• Extending the capabilities of the Roboflow 100 auto-benchmark server
• Training a YOLOv8 Objects 365 checkpoint on YOLOv8 and evaluating how much it improves Roboflow 100 training results
• Writing a blog and example training notebook on DETA
• Assessing and implementing an auto-batching strategy for our training machines
• Surveying general open source OCR models and implementing one of them in the Roboflow inference server
• Assessing and implementing Neural Magic apis to deploy Roboflow models
• Training a general object detector across on Roboflow Universe datasets
• Evaluating capabilities of zero-shot object detection models for label assist