Senior Azure Data Engineering Consultant

Key Skills

Data Visualization, Databricks, Docker, Effective communication skills, ETL/ELT technologies, Google Dataflow, PySpark, Python Programming, RESTful, SoapUI, SQL, Teradata, XML-JSON

Job Description

ProCogia has experienced considerable growth over the last two years & core to our culture is maintaining a high-level of gender equality throughout the company. Our diversity and differences allow us to create innovative and effective data solutions for our clients.

Our Mission

We enable our customers to make intelligent and agile decisions through strategically tailored data solutions. We attract, develop and retain diverse, motivated and collaborative team players who love what they do. 

Our Values

TRUST We build trust in all of our relationships, internally and externally

GROWTH We believe in life-long learning; both personally and professionally

INNOVATION Our technology-agnostic & innovative approach enables cutting-edge data solutions

EXCELLENCE We are committed to delivering excellence in everything we do

ProCogia Overview

At ProCogia we’re passionate about developing data-driven solutions that provide highly informed answers to our clients’ most critical challenges. Our projects are varied, from Data Warehouse builds, deploying Cloud Data Solutions, Dashboarding, & building predictive models.

We work with industry leading clients from various sectors including Pharmaceuticals, Telecommunications, Technology, Financial Services & Retail. Our work environment ensures opportunities to gain valuable experience in various industries enhancing your personal & career development. 

ProCogia has doubled in size over the last two years & core to ProCogia’s culture is ensuring we maintain a balanced gender ratio. Our diversity, and differences allow us to create innovative and effective solutions for our clients. 

Job Description

The Senior Data Engineer has the following responsibilities and duties: 

• Create and maintain optimal data architecture pipeline.
• Perform root cause analysis on internal and external data, processes and share insights with different stakeholders using reporting and visualizations tools.
• Design infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources into Snowflake and other platforms
• Understand existing implementation and should enhance it as needed.
• Work with other data engineers, data ingestion specialists, and experts across cross-functional teams.
• Work on large datasets to meet functional and business requirements

Requirements

The Senior Data Engineering Consultant should have the following skills, education, and experience: 

Skills

Required:

• Strong experience with Snowflake Data Warehouse
• Excellent SQL and Python, PySpark skills using Spark framework.
• Strong knowledge across Microsoft Azure cloud services to build scalable data pipelines and workflow management using ADF, Dataflow , usage of Databricks with PySpark
• Experience with ETL/ELT and familiarity of Data Warehouse concepts
• Expertise in designing and working with efficient Data Models in big data ecosystem.
• Experience with Docker/containerization
• Knowledge of API-based integration and security (Restful and SOAP)
• Experience with Data Visualization tools such as Power BI etc.
• Ability to learn fast and translate data into actionable results
• Excellent communication skills with onsite team and client stakeholders

Preferred:

• Experience with JSON Programming
• Solid Linux programming skills
• Cosmos DB, Teradata, SQL Server (all nice to have)
• Stream-processing systems: Streaming-Analytics, IoT Hub, Event Hub, Kusto Queries (all nice to have)

Education

Required:

• Bachelors in a quantitative field such as computer science, computer engineering other related disciplines.

Preferred:

• Masters or PhD in a quantitative field such as computer science, computer engineering other related disciplines. 

Experience

• An experienced professional will have at least 5+ years of professional experience in Data Engineering role with a Masters Degree or 7+ years of experience in a Data Engineering related role with a Bachelors degree