Senior Software Engineer, Model Serving

Key Skills

Aartificial intelligence, API, AWS, Deep Learning, Google Cloud Platform (GCP), GPUs, High performance computing- HPC, Large Language Models - LLMs, Machine learning techniques, Natural Language Processing (NLP)

Job Description

We’re ambitious — we believe our technology will fundamentally transform how industries interact with natural language. And we have the technical chops to back it up - Cohere’s CEO, Aidan Gomez, is a co-author of the groundbreaking paper “Attention is all you need”, and was previously part of Google Brain. Our entire technical team is world-class. 

We are focused on creating a diverse and inclusive work environment so that all of our team members can thrive. We welcome kind and brilliant people to our team, from wherever they come.

Why this role?

Are you energized by building high-performance, scalable and reliable machine learning systems? Do you want to help define and build the next generation of AI platforms powering advanced NLP applications? We are looking for Senior Software Engineers to join the Model Serving team at Cohere. The team is responsible for developing, deploying, and operating the AI platform delivering Cohere's large language models through easy to use API endpoints. In this role, you will work closely with many teams to deploy optimized NLP models to production in low latency, high throughput, and high availability environments. You will also get the opportunity to interface with customers and create customized deployments to meet their specific needs.

We are looking for candidates with a range of experiences for multiple roles, from senior to staff-level engineers.

Please Note: We have offices in Toronto, Palo Alto, and London but embrace being remote-first! There are no restrictions on where you can be located for this role.

Must Have Qualifications:

• Professional software development experience with a strong understanding of core principles and best practices
• Experience in Golang (or, other languages designed for high-performance scalable servers)
• Ability to guesstimate system behavior and resource requirements under various conditions

Preferred Qualifications:

• Experience with serving ML models
• Experience designing, implementing, and maintaining a production service at scale
• Familiarity with inference characteristics of deep learning models, specifically, Transformer based architectures.
• Familiarity with computational characteristics of accelerators (GPUs, TPUs, and/or Inferentia), especially how they influence latency and throughput of inference.
• Strong understanding or working experience with distributed systems
• Experience in performance benchmarking, profiling, and optimization.
• Experience with cloud infrastructure (e.g. AWS, GCP)

This job posting was partly written using Cohere’s large language models.

If some of the above doesn’t line up perfectly with your experience, we still encourage you to apply! If you consider yourself a thoughtful worker, a lifelong learner, and a kind and playful team member, Cohere is the place for you.

We value and celebrate diversity and strive to create an inclusive work environment for all. We welcome applicants of all kinds and are committed to providing an equal opportunity process. Cohere provides accessibility accommodations during the recruitment process. Should you require any accommodation, please let us know and we will work with you to meet your needs.