-
Job Type
Full Time -
Experience
6-8 year -
Salary
₹90,000 - ₹250,000 / Month -
Location
Bangalore, Karnataka, India -
Job Function
-
Industry
Information Technology -
Qualification
Degree in Artificial intelligence- AI
Key Skills
Aartificial intelligence, Apache Hadoop, Apache Kafka, Argo, AWS, Azure, Big Data Technology, Data pipelines, Data security, DevOps, Docker, Effective communication skills, Google Cloud Platform (GCP), Java Programming, Machine learning techniques, MLOps tools, Python Programming, PyTorch, Red Hat OpenShift, Scala Programming, Scikit-learn, SPARK Programming, TensorFlow
Job Description
Your Role and Responsibilities
We are looking for a highly skilled and experienced Senior Machine Learning and Data Engineer to join our team of AI experts. You will be responsible for designing, implementing and optimizing our data and machine learning pipelines across a wide variety of applications. The ideal candidate is an experienced data and ML pipeline builder who enjoys optimizing data and machine learning systems and building them from the ground up. The ML Data Engineer will support our software developers and data scientists in embedding AI into our products and will ensure that our data and ML delivery architecture is consistent across projects. They must be self-directed and comfortable supporting the data needs of multiple teams, systems and products. The right candidate will be excited by the prospect of optimizing and re-designing our data and ML architecture to support our next generation of products and data initiatives.
Key Responsibilities:
- Design and develop robust, scalable, and high-performance data and machine learning pipelines, ensuring data integrity and reliability throughout the process.
- Collaborate with cross-functional teams, including data scientists, product managers, and software engineers, to understand data and machine learning requirements and deliver optimal solutions.
- Experience delivering and maintaining productionized end-to-end Machine Learning solutions, form data preparation, experimentation, model training, model serving, and model update
- Design solutions that ensuring efficient data storage, retrieval, and processing capabilities using modern data lakehouse and data fabric systems.
- Develop and implement data transformation and cleansing processes to ensure data quality and consistency.
- Optimize and fine-tune data and machine learning pipelines for improved performance, scalability, and reliability.
- Collaborate with the DevOps team to ensure data security, privacy, and compliance with relevant regulations and best practices.
- Document technical specifications, system architecture, and data flows to facilitate knowledge sharing and maintain a well-documented system.
- Stay updated with emerging technologies, industry trends, and best practices in data engineering, and proactively introduce new tools, frameworks, and techniques to enhance data processing capabilities.
Required Technical and Professional Expertise
- 5+ years as a Data and Machine Learning Engineer or similar role with a focus on designing and developing end-to-end enterprise data and machine learning pipelines
- Knowledge of data best practices and MLOps/Dev operations in SaaS and hybrid environments
- 5+ years of Python, Scala or Java development experience
- Ability in managing and communicating data and machine learning pipeline plans to internal clients
- Experience building, running, and maintaining productionized Machine Learning models using a well-designed MLOps approach
- Experience working with both batch and streaming data
- Experience with machine learning frameworks like PyTorch, Tensorflow and scikit-learn
- Experience with cloud-based platforms and services (e.g., IBM, AWS, Azure, Google Cloud).
- Strong expertise in working with big data technologies and frameworks like Hadoop, Spark, or Kafka.
- Strong understanding of data governance, data security, and data privacy best practices.
- Excellent communication skills with the ability to effectively collaborate with technical and non-technical stakeholders.
Preferred Technical and Professional Expertise
- Experience designing, building, and maintaining data processing and machine learning systems working in containerized environments (Docker, OpenShift, k8s)
- Experience working with both batch and streaming data ingestion
- Experience building reusable and reliable workflows and pipelines using workflow engines (Argo, Tekton, etc.)
- Experience developing or leveraging automated platforms for model observability, model quality and drift and systems to automatically identify and correct issues.
- Experience developing or leveraging automated platforms for data observability, data quality and drift and systems to automatically identify and correct data issues.
Company Info.
IBMIBM is a leading cloud platform and cognitive solutions company. Restlessly reinventing since 1911, we are the largest technology and consulting employer in the world, with more than 290,000 employees serving clients in 177 countries. IBM Research provides unparalleled insight into business, industry and society by leveraging advanced computing architectures and methodologies to solve some of the world’s most pressing challenges.
-
IndustryInformation Technology,Computer software,Computer hardware
-
No. of Employees292,500
-
LocationNew Orchard Road, Armonk, New York, NY 10504, USA
-
Website
-
Jobs Posted
Get Similar Jobs In Your Inbox
IBM is currently hiring Data Engineer Jobs in Bangalore, Karnataka, India with average base salary of ₹90,000 - ₹250,000 / Month.
Similar Jobs View More
Industrial Consulting Senior Manager: Analytics & AI
IBMResearch Triangle Park, Durham, NC, USA
2-4 year
Entry Level - Associate Data Scientist (Commercial)
IBMResearch Triangle Park, Durham, NC, USA
2-4 year
2023 Internship - Discovery Science & Applications Research Scientist
IBMCambridge, MA, USA
2-4 year
Postdoctoral Researcher – Quantum Applications and Molecular Dynamics
IBMSan Jose, CA, USA
2-4 year
Spanish Natural Language Processing Dialog Engineer – Watson Orders
IBMMountain View, CA, USA
2-4 year
Master's Internship in Software Engineering Human-AI Interaction
IBMRio de Janeiro, Brazil
0-2 year
Master's Internship in Software Engineering Human-AI Interaction
IBMHortolândia, State of São Paulo, Brazil
0-2 year
Research Software Engineering Internship on Human-AI Interaction
IBMRio de Janeiro, Brazil
0-2 year
Master's Internship in Research in Software Engineering
IBMHortolândia, State of São Paulo, Brazil
0-2 year
2024 Intern: Research Scientist – AI-powered Automation
FEATURED IBMYorktown Heights, NY, USA
0-2 year
Intern: RS – Foundation Models for Data Management & Lakehouses
IBMYorktown Heights, NY, USA
0-2 year
Intern: RS – Foundation Models for Data Management & Lakehouses
FEATURED IBMCambridge, MA, USA
0-2 year
Machine Learning Developer Intern (May – 4 months) – IBM Consulting
FEATURED IBMToronto, ON, Canada
0-2 year
Machine Learning Developer Intern (May – 4 months) – IBM Consulting
IBMCalgary, AB, Canada
0-2 year
Machine Learning Developer Intern (May – 4 months) – IBM Consulting
IBMMontreal, QC, Canada
0-2 year
AI & Cryptography – MSc & PhD Student Positions – Israel Research Lab
IBMGivatayim, Israel
0-2 year
Business Transformation Consultant – Business Intelligence, german
IBMCluj-Napoca, Romania
2-4 year
Data & AI Sales Specialist – Atlanta Market/Southeast
IBMResearch Triangle Park, Durham, NC, USA
4-6 year
AI Speech Technologies- MSc/PhD Summer internship 2024 – Research Lab
IBMGivatayim, Israel
0-2 year
LLM-NLP Intelligent Automation-MSc&PhD Summer intern 2024-Research Lab
IBMGivatayim, Israel
0-2 year
AI Intelligent Automation-MSc&PhD Summer internship 2024-Research Lab
IBMGivatayim, Israel
0-2 year
AI for Drug Discovery- MSc & PhD Summer internship 2024- Research Lab
IBMGivatayim, Israel
0-2 year
Research Software Engineer for Cloud-based AI Platforms
IBMSão Paulo, State of São Paulo, Brazil
2-4 year
Research Software Engineer for Cloud-based AI Platforms
IBMRio de Janeiro, State of Rio de Janeiro, Brazil
2-4 year
Research Software Engineer for AI Accelerator’s Software Stack
IBMSão Paulo, State of São Paulo, Brazil
2-4 year
Research Software Engineer for AI Accelerator’s Software Stack
IBMRio de Janeiro, State of Rio de Janeiro, Brazil
2-4 year
IBM Brazil Internship Program – Associates
IBMRio de Janeiro, State of Rio de Janeiro, Brazil
0-2 year
Associate Business Transformation Consultant – Data Platforms (w/m/x)
IBMFrankfurt, Germany
4-6 year
Associate Business Transformation Consultant – Data Platforms (w/m/x)
IBMEhningen, Germany
4-6 year
Technical Specialist - Data & AI Applications
IBMAustin, TX, USA; Chicago, IL, USA; Dallas, TX, USA; Houston, TX, USA; Los Angeles, CA, USA; New York, NY, USA; Philadelphia, PA, USA; Phoenix, AZ, USA; San Antonio, TX, USA; San Diego, CA, USA; San Jose, CA, USA
2-4 year
Technical Sales Specialist (Data Visualization and AI Apps) - Midwest
IBMAustin, TX, USA; Chicago, IL, USA; Dallas, TX, USA; Houston, TX, USA; Los Angeles, CA, USA; New York, NY, USA; Philadelphia, PA, USA; Phoenix, AZ, USA; San Antonio, TX, USA; San Diego, CA, USA; San Jose, CA, USA
2-4 year
Hybrid Data Management - Technical Specialist
IBMAustin, TX, USA; Chicago, IL, USA; Dallas, TX, USA; Houston, TX, USA; Los Angeles, CA, USA; New York, NY, USA; Philadelphia, PA, USA; Phoenix, AZ, USA; San Antonio, TX, USA; San Diego, CA, USA; San Jose, CA, USA
2-4 year
Data Scientist - IBM Client Engineering - Midwest 1
IBMChicago, IL, USA; Columbus, OH, USA; Detroit, MI, USA; Indianapolis, IN, USA; Kansas City, MO, USA; Milwaukee, WI, USA; Minneapolis, MN, USA; Omaha, NE, USA
2-4 year