Azure Data engineer

Key Skills

Aartificial intelligence, Analytical and Problem solving, Azure, Big Data Technology, Data Engineering, Data pipelines, Data science techniques, Data Warehousing, Databricks, DevOps, Machine learning techniques, Python Programming, T-SQL

Job Description

The Role

Roles and Responsibilities:

Work as part of a team to develop Cloud Data and Analytics solutions

Strong experience as an Azure Data Engineer and must have Azure Databricks experience.

Demonstrated experience of turning business use cases and requirements into technical solutions.

Expert level understanding on Azure Data Factory, SQL/Synapse , ADLS, Azure Databricks, Pyspark, Azure DevOps, Python is required.

Experience in business processing mapping of data and analytics solutions.

Ability to conduct data profiling, cataloguing, and mapping for technical design and construction of technical data flows.

The ability to apply such methods to solve business problems using one or more Azure Data and Analytics services in combination with building data pipelines, data streams, and system integration.

Postgress DB, Azure Purview , Kusto Query, Azure Cosmos DB, ADX is a plus.

Experience preparing data for Data Science and Machine Learning.

Experience preparing data for use in Azure Machine Learning and/or Azure Databricks .

Demonstrated experience preparing data and building data pipelines for AI Use Cases (text, voice, image, etc.…).

Designing and building Data Pipelines using streams of IoT data.

Knowledge of Lambda and Kappa architecture patterns.

Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code is essential.

Strong team collaboration and experience working with remote teams.

Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code fundamentals.

Working experience with Visual Studio, PowerShell Scripting, and ARM templates.

Experience with Git/TFS/VSTS is a must.

Who You Are

Required Professional and Technical Expertise:

7+ years experience with Azure Data Engineering

Hands-on experience in Azure Databricks – Mandatory

Experience as Data Engineer in Azure Big Data Environment

Expertise in ETL tools i.e. (SSIS, Data Stage)

Expertise in Implementing Data Warehousing Solutions

Experience with working in Agile (KANBAN, Scrumban) environment as part of a scrum team

Programming experience in Scala or Python, T-SQL

Hands-on experience in Azure stack (Azure Data Lake, Azure Data Factory - Optional

Good understanding of Azure Databricks platform and ability to build data analytics solutions to support the required performance & scale

Good understanding and experience working on Deltalake using Apache Delta in Azure Databricks

Experienced working on performance tuning and optimizing long running jobs

Demonstrated analytical and problem-solving skills, particularly those that apply to a big data environment.

Good Understanding of Modern Data Warehouse and Data Warehousing concepts

Proficient in a source code control system (e.g. CodeCloud)

Required Education:

Bachelor’s degree in computer science, Electrical/Electronic Engineering, Information Technology or another related field

Being You

Diversity is a whole lot more than what we look like or where we come from, it’s how we think and who we are. We welcome people of all cultures, backgrounds, and experiences. But we’re not doing it single-handily: Our Kyndryl Inclusion Networks are only one of many ways we create a workplace where all Kyndryls can find and provide support and advice. This dedication to welcoming everyone into our company means that Kyndryl gives you – and everyone next to you – the ability to bring your whole self to work, individually and collectively, and support the activation of our equitable culture. That’s the Kyndryl Way.