Specialist - Data Engineering

Key Skills

Agile methodologies, API, Azure, Continuous Integration & Continuous Delivery - CI/CD, Data pipelines, Data Warehousing, Docker, ETL/ELT technologies, Github, InfluxDB, Kubernetes-K8s, PowerShell Programming, PySpark, Python Programming, R Programming, Scala Programming

Job Description

Responsibilities:

• Design and implement highly available and scalable Azure cloud data solutions that meet business requirements.

• Collaborate with cross-functional teams to design and implement solutions that meet business requirements.

• Participates in the testing process through test review and analysis.

• Developing and maintaining data lake and data warehouse schematics, layouts, architectures and relational/non-relational databases for data access and Advanced Analytics.

Education

• Bachelor’s degree in Computer Science, Applied Mathematics, Data Science, or Machine Learning.

Required Qualifications:

 5+ years of experience as an Azure data engineer

 Strong experience with Azure cloud technologies say- Azure Data Factory, Azure Data Bricks, Cosmos DB, Mongo db, Eventhub, Servicebus, Topic, blob storages, data lakes, azure functions, AKS, redis cache, keyvault, Iothub, webjobs

 Experience with data transformation and manipulation using Azure Databricks or similar tools

 Experience with streaming data platforms – Kafka

 Experience in dockers, Kubernetes, Microservices

 Solid scripting experience with Python, PowerShell, Pyspark, R, Scala

• Good working knowledge with Azure Infrastructure, security, token management, service principals, managed identity, authentication, authorization management

• Experience on data integrations with API's.

• Experience working in an Agile development environment

• Familiar with a variety of the field’s concepts, practices, and procedures and a strong focus on data integrity.

• Demonstrates strong written and oral communications skills. Ability, desire, and focus to meet deadlines Demonstrates ability to work with all levels of individuals.

• Demonstrates organizational skills while working on multiple projects; and communicate effectively within the team.

• Working knowledge of CI/CD tools and concepts - Azure Pipelines, Github actions &Workflow,

• Architect data ingestion solutions such as ETL, ELT’s on a wide set of data storage solutions.

• Flexible to learn on new programming languages/technologies based on project needs

• Flexible to learn & work with other cloud platform aws, google

• Independent and able to manage and prioritize workload.

• Ability to guide or lead junior resources to get desired project results.

• Advanced troubleshooting skills to drive to root cause

• Ability to adapt quickly and positively to change.

• Ability to manage ambiguity and solve undefined problems

Nice to Have:

• Good knowledge of Timeseries databases (eg; InfluxDB, TimescaleDB)

• Observability expertise with Dynatrace is add-on

• Go language expertise is add-on

• IIOT domain expertise is add-on

Job Segment: Computer Science, Machinist, Data Warehouse, Engineer, Warehouse, Manufacturing, Technology, Engineering