Principal Applied AI Scientist (LLMs) (Hybrid)

Key Skills

Aartificial intelligence, AWS, Computer Vision (CV), Data science techniques, Google Cloud Platform (GCP), Infrastructure as code, Kubernetes-K8s, Large Language Models - LLMs, Machine learning techniques, MLOps tools, Python Programming, User support and Troubleshooting

Job Description

The Opportunity

In this role, you will be able to make a real impact on the safety of large language models and generative AI solutions across different verticals. You will work on the cutting edge of envisioning and building new types of tools and algorithms to monitor, explain, and improve such models.

What You'll Do

• As a Principal Applied AI Scientist, you will play a key role in shaping the future of Fiddler’s observability solutions for Large Language Models (LLMs) and Generative AI. You will develop and tune novel LLMs, ML models, and other techniques to monitor, classify, and protect large language model prompts/responses and related use cases.
• Establish thought-leadership by following up on state of the art research in Large Language models, Gen AI and responsible AI. Conduct experiments, build POCs, evaluate and benchmark novel techniques related to LLM monitoring, debugging, guardrails etc. Identify ways to leverage some of these into product offerings and guide efforts to pursue these.
• Coordinate with multinational teams to drive projects from research POC to production.
• Develop deep product understanding. Think critically and creatively to recognize common customer challenges and craft powerful generalizable solutions.
• Support Sales and Customer Success teams to address some of the more advanced customer questions and asks that require deep understanding of novel ML use-cases. Provide expert guidance to customers with challenging explainability uses.
• Author white papers, blog posts, and develop technical content related to ML and the Fidder platform.

What We are Looking For

• MS/PhD in Computer Science, Data Science or equivalent discipline.
• 8-10 years of industry experience.
• Deep technical understanding of Large language models. Experience in building, training and fine tuning LLMs . Practical experience with the latest technologies in LLM and generative AI, such as parameter-efficient fine-tuning, instruction fine-tuning, and advanced prompt engineering techniques like Tree-of-Thought / Chain-of-Thought.
• Experience setting up, monitoring and tuning ML models and Large language models in production. Experience with common LLM frameworks (Langchain, Llamaindex, RAG, HuggingFace, BERT models, quantization, and eval frameworks like Ragas or Presidio) and MLOps best practices.
• You invest in staying up to date with the LLM space, you regularly educate yourself on new capabilities, market trends and needs and are involved in local user groups and/or events relating to LLMs. You have experience speaking at relevant industry events.
• Proficiency with Python, ability to write production grade code and support ML model deployments that can handle live workloads.
• Ability to conduct deep analysis and benchmarking of models and methods.
• Experience debugging and troubleshooting production systems

Nice to Have

• Experience working with Computer vision models.
• Experience with data visualization and dimensionality reduction techniques.
• Knowledge of infrastructure management (AWS / Google Cloud, Kubernetes, or equivalent on-premises technologies)

Pay Range in San Francisco California is $250,000 - $320,000. Hybrid with at least 2 days a week in the Palo Alto office, usually Mondays and Thursdays.