Senior Data Engineer Consultant

Key Skills

Amazon Athena, Amazon RedShift, Amazon Simple Storage Service (S3), Apache Airflow, AWS Glue, Data Visualization, Databricks, Docker, Effective communication skills, ETL/ELT technologies, Google Dataflow, Lambda, PySpark, Python Programming, RESTful, SoapUI, SQL, Teradata, XML-JSON

Job Description

ProCogia has experienced considerable growth over the last two years & core to our culture is maintaining a high-level of gender equality throughout the company. Our diversity and differences allow us to create innovative and effective data solutions for our clients.

Our Mission

We enable our customers to make intelligent and agile decisions through strategically tailored data solutions. We attract, develop and retain diverse, motivated and collaborative team players who love what they do. 

Our Values

TRUST We build trust in all of our relationships, internally and externally

GROWTH We believe in life-long learning; both personally and professionally

INNOVATION Our technology-agnostic & innovative approach enables cutting-edge data solutions

EXCELLENCE We are committed to delivering excellence in everything we do

ProCogia Overview

At ProCogia we’re passionate about developing data-driven solutions that provide highly informed answers to our clients’ most critical challenges. Our projects are varied, from Data Warehouse builds, deploying Cloud Data Solutions, Dashboarding, & building predictive models.

We work with industry leading clients from various sectors including Pharmaceuticals, Telecommunications, Technology, Financial Services & Retail. Our work environment ensures opportunities to gain valuable experience in various industries enhancing your personal & career development. 

ProCogia has doubled in size over the last two years & core to ProCogia’s culture is ensuring we maintain a balanced gender ratio. Our diversity, and differences allow us to create innovative and effective solutions for our clients. 

Job Description

The Senior Data Engineer – AWS Cloud has the following responsibilities and duties: 

• Responsible for designing, deploying, and managing high quality data solutions in the AWS cloud ecosystem.
• Create and maintain optimal data architecture pipeline.
• Collaborate with various onsite and client stakeholders in Agile environment to identify data engineering requirements and implement data solutions.
• Strive for continuous improvement of code quality and development practices ensuring Data services are reliable and scalable.
• Collaborate with other data engineers, data Analysts across cross-functional teams and provide expert guidance to junior members on best practices for data engineering in AWS cloud.
• Perform root cause analysis on internal and external data, processes and share insights with different stakeholders using reporting and visualizations tools.
• Work on large datasets to meet functional and business requirements.

The Senior Data Engineer – AWS Cloud should have the following skills, education, and experience: 

Skills

Required:

• Experience across AWS cloud services to build scalable data pipelines and data engineering workflow
• Experience with S3, EC2, EMR, Lambda, Glue, Athena, Airflow, Redshift.
• Strong experience with SQL and Python, PySpark skills using Spark framework.
• Experience with enterprise Data warehousing implementations using Batch and real time systems.
• Expertise in working with efficient Data Models in a big data ecosystem.
• Experience with Docker/containerization, Knowledge of IAM roles, Policies, etc.
• Knowledge of API-based integration and security (Restful and SOAP)
• Experience with Data Visualization tools such as Power BI, Tableau etc.
• Ability to learn fast and translate data into actionable results
• Excellent communication skills with onsite team and client stakeholders
• Familiar with EKS and ECS AWS services.

Preferred:

• Experience with DataBricks preferred.
• Solid Linux programming skills
• DynamoDB, Teradata, SQL Server.
• Familiar with AWS Devops, 
• Stream-processing systems: Streaming-Analytics
• Knowledge of Kinesis Data Streams, Kinesis Firehose, etc. 
• Experience with Snowflake Data Warehouse preferred.

Education

Required:

• Bachelors in a quantitative field such as computer science, computer engineering other related disciplines.

Preferred:

• Masters or PhD in a quantitative field such as computer science, computer engineering other related disciplines. 

Experience

• An experienced professional will have at least 3+ years of professional experience in Data Engineering role with a Masters Degree or 5+ years of experience in a Data Engineering related role with a Bachelors degree