Data Engineer - Elastic

Key Skills

AWS, Azure, Data Engineering, Database, Docker, Elasticsearch, Google Cloud Platform (GCP), IT, Java Programming, Kibana, Kubernetes-K8s, Machine learning techniques, NoSQL, Python Programming, SQL

Job Description

As an ELK (Elastic, Logstash & Kibana) Data Engineer, you would be responsible for developing, implementing, and maintaining the ELK stack-based solutions for Kyndryl’ s clients. This role would be responsible to develop efficient and effective, data & log ingestion, processing, indexing, and visualization for monitoring, troubleshooting, and analysis purposes.

Education, Experience, and Certification Requirements:

BS or MS degree in Computer Science or a related technical field 5+ years overall IT Industry Experience. 3+ years of development experience with Elastic, Logstash and Kibana in designing, building, and maintaining log & data processing systems 3+ years of Python or Java development experience 4+ years of SQL experience (No-SQL experience is a plus) 4+ years of experience with schema design and dimensional data modelling Experience working with Machine Learning model is a plus Knowledge of cloud platforms (e.g., AWS, Azure, GCP) and containerization technologies (e.g., Docker, Kubernetes) is a plus “Elastic Certified Engineer” certification is preferrable

Key Responsibilities:

• Configure Logstash to receive, filter, and transform logs from diverse sources (e.g., servers, applications, AppDynamics, Storage, Databases and so son) before sending them to Elasticsearch.
• Configure ILM policies, Index templates etc.
• Develop Logstash configuration files to parse, enrich, and filter log data from various input sources (e.g., APM tools, Database, Storage and so on)
• Implement techniques like grok patterns, regular expressions, and plugins to handle complex log formats and structures.
• Ensure efficient and reliable data ingestion by optimizing Logstash performance, handling high data volumes, and managing throughput.
• Utilize Kibana to create visually appealing dashboards, reports, and custom visualizations.
• Collaborate with business users to understand their data integration & visualization needs and translate them into technical solutions
• Establishing the correlation within the data and develop visualizations to detect the root cause of the issue.
• Integration with ticketing tools such as Service Now
• Hands on with ML and Watcher functionalities
• Monitor Elasticsearch clusters for health, performance, and resource utilization
• Create and maintain technical documentation, including system diagrams, deployment procedures, and troubleshooting guides

Who You Are

BS or MS degree in Computer Science or a related technical field 5+ years overall IT Industry Experience. 3+ years of development experience with Elastic, Logstash and Kibana in designing, building, and maintaining log & data processing systems 3+ years of Python or Java development experience 4+ years of SQL experience (No-SQL experience is a plus) 4+ years of experience with schema design and dimensional data modelling Experience working with Machine Learning model is a plus Knowledge of cloud platforms (e.g., AWS, Azure, GCP) and containerization technologies (e.g., Docker, Kubernetes) is a plus “Elastic Certified Engineer” certification is preferrable

Being You

Diversity is a whole lot more than what we look like or where we come from, it’s how we think and who we are. We welcome people of all cultures, backgrounds, and experiences. But we’re not doing it single-handily: Our Kyndryl Inclusion Networks are only one of many ways we create a workplace where all Kyndryls can find and provide support and advice. This dedication to welcoming everyone into our company means that Kyndryl gives you – and everyone next to you – the ability to bring your whole self to work, individually and collectively, and support the activation of our equitable culture. That’s the Kyndryl Way.

What You Can Expect

With state-of-the-art resources and Fortune 100 clients, every day is an opportunity to innovate, build new capabilities, new relationships, new processes, and new value. Our employee learning hub gives you access to the best learning in the industry to receive certifications and accreditations, including Microsoft University, AWS Cloud Center of Excellence, Udemy, and the Harvard Business Review. Through our company-wide volunteering and giving platform, you can donate, start fundraisers, volunteer, and search over 2 million non-profit organizations. At Kyndryl, we invest heavily in you! We want you to succeed so that together, we will all succeed.