Senior Software Engineer (MLOps) – Training & Registry

Key Skills

Design, Kubernetes-K8s, Machine learning techniques, MLflow, MLOps tools, Python Programming

Job Description

What You’ll Do:

• Design and implement scalable, reliable systems for training orchestration, artifact tracking, and model registration across multiple data centers and cloud regions.
• Improve and streamline ML experimentation workflows by integrating tooling like Ray, Airflow, and interactive notebooks.
• Develop APIs and services that enable applied scientists to seamlessly launch, debug, and track training jobs.
• Ensure reproducibility and traceability by building robust version control and metadata systems for model artifacts.
• Collaborate with AI infra teams (LLMObs, Compute, etc.) to deliver consistent user experiences and integrated telemetry.
• Mentor engineers and help drive architectural decisions and technical standards.

Who You Are:

• You have 6+ years of experience in backend, distributed systems, or platform engineering roles.
• You have worked on ML platforms or infrastructure, ideally supporting real-world training or model lifecycle workflows.
• You’re comfortable designing APIs, managing data at scale, and architecting systems for reliability and observability.
• You’re fluent in Python or Go and have experience with cloud-native tools (e.g., Kubernetes, object stores, queueing systems).
• You’re comfortable navigating cross-functional environments and translating scientific requirements into reliable systems.
• Bonus points: experience with model registries, experiment tracking tools (e.g., MLflow, Weights & Biases), or distributed training frameworks.

Datadog values people from all walks of life. We understand not everyone will meet all the above qualifications on day one. That's okay. If you’re passionate about technology and want to grow your skills, we encourage you to apply.

Benefits and Growth:

• New hire stock equity (RSUs) and employee stock purchase plan (ESPP)
• Continuous professional development, product training, and career pathing
• Intradepartmental mentor and buddy program for in-house networking
• An inclusive company culture, ability to join our Community Guilds (Datadog employee resource groups)
• Access to Inclusion Talks, our internal panel discussions
• Free, global mental health benefits for employees and dependents age 6+
• Competitive global benefits

Benefits and Growth listed above may vary based on the country of your employment and the nature of your employment with Datadog.