3D data processing, Aartificial intelligence, Big Data Technology, Large Language Models - LLMs, Machine learning techniques, NeurIPS, Python Programming, SPARK Programming
IBM Research Engineers are charting the future of Artificial Intelligence, creating breakthroughs in quantum computing, discovering how blockchain will reshape the enterprise, and much more. Join a team that is dedicated to applying science to some of today’s most complex enterprise challenges, whether it’s building state of art LLMs, discovering new quantum algorithms and applications or advances in multi cloud computers.
Your Role and Responsibilities
We are seeking an engineer to refine and standardize our large language model training procedures. This role is crucial for enhancing our existing framework, encompassing a broad spectrum of tasks from dataset preprocessing to large-scale distributed training techniques such as fine-tuning, contrastive fine-tuning, instruct tuning, etc. The candidate will be instrumental in developing tools and scripts that streamline the entire alignment pipeline, ensuring efficiency and consistency.
The engineer will be responsible for supporting all tooling and software engineering efforts to standardize and optimize the instruction tuning pipeline. They should possess the ability to interpret machine learning research and translate it into reliable, maintainable software. Additionally, the role involves developing specialized pipelines for various tasks, such as building RLHF pipelines with unit test-based rewards for low-resource languages and creating effective LLM-generated data pipelines. The ideal candidate will have a passion for programming languages, enabling them to tailor data generation and training pipelines to specific languages, and if applicable, leverage webapp development skills for creating frameworks to collect human data and deploy models in user-centric platforms.
Required Technical and Professional Expertise
Preferred Technical and Professional Expertise
IBM is a leading cloud platform and cognitive solutions company. Restlessly reinventing since 1911, we are the largest technology and consulting employer in the world, with more than 290,000 employees serving clients in 177 countries. IBM Research provides unparalleled insight into business, industry and society by leveraging advanced computing architectures and methodologies to solve some of the world’s most pressing challenges.
Research Triangle Park, Durham, NC, USA
2-4 year
Research Triangle Park, Durham, NC, USA
2-4 year
Cambridge, MA, USA
2-4 year
San Jose, CA, USA
2-4 year
Mountain View, CA, USA
2-4 year
Rio de Janeiro, Brazil
0-2 year
Hortolândia, State of São Paulo, Brazil
0-2 year
Rio de Janeiro, Brazil
0-2 year
Hortolândia, State of São Paulo, Brazil
0-2 year
Yorktown Heights, NY, USA
0-2 year
Yorktown Heights, NY, USA
0-2 year
Cambridge, MA, USA
0-2 year
Toronto, ON, Canada
0-2 year
Calgary, AB, Canada
0-2 year
Montreal, QC, Canada
0-2 year
Givatayim, Israel
0-2 year
Cluj-Napoca, Romania
2-4 year
Research Triangle Park, Durham, NC, USA
4-6 year
Givatayim, Israel
0-2 year
Givatayim, Israel
0-2 year
Givatayim, Israel
0-2 year
Givatayim, Israel
0-2 year
São Paulo, State of São Paulo, Brazil
2-4 year
Rio de Janeiro, State of Rio de Janeiro, Brazil
2-4 year
São Paulo, State of São Paulo, Brazil
2-4 year
Rio de Janeiro, State of Rio de Janeiro, Brazil
2-4 year
Rio de Janeiro, State of Rio de Janeiro, Brazil
0-2 year
Frankfurt, Germany
4-6 year
Ehningen, Germany
4-6 year
Austin, TX, USA; Chicago, IL, USA; Dallas, TX, USA; Houston, TX, USA; Los Angeles, CA, USA; New York, NY, USA; Philadelphia, PA, USA; Phoenix, AZ, USA; San Antonio, TX, USA; San Diego, CA, USA; San Jose, CA, USA
2-4 year
Austin, TX, USA; Chicago, IL, USA; Dallas, TX, USA; Houston, TX, USA; Los Angeles, CA, USA; New York, NY, USA; Philadelphia, PA, USA; Phoenix, AZ, USA; San Antonio, TX, USA; San Diego, CA, USA; San Jose, CA, USA
2-4 year
Austin, TX, USA; Chicago, IL, USA; Dallas, TX, USA; Houston, TX, USA; Los Angeles, CA, USA; New York, NY, USA; Philadelphia, PA, USA; Phoenix, AZ, USA; San Antonio, TX, USA; San Diego, CA, USA; San Jose, CA, USA
2-4 year
Chicago, IL, USA; Columbus, OH, USA; Detroit, MI, USA; Indianapolis, IN, USA; Kansas City, MO, USA; Milwaukee, WI, USA; Minneapolis, MN, USA; Omaha, NE, USA
2-4 year