Data Engineering Lead

Key Skills

Analytical and Problem solving, Apache Airflow, Big Data Technology, Continuous Integration & Continuous Delivery - CI/CD, Data Analytics, Data Engineering, Data science techniques, Data Transformation, Docker, Employee life cycle, ETL/ELT technologies, Kubernetes-K8s, Leadership Skill, PowerBI, PySpark, Python Programming, SQL

Job Description

The next step of your career starts here, where you can bring your own unique mix of skills and perspectives to a fast-growing team.

Metyis is a global and forward-thinking firm operating across a wide range of industries, developing and delivering Big Data, Digital Commerce, Marketing & Design solutions and Advisory services. At Metyis, our long-term partnership model brings long-lasting impact and growth to our business partners and clients through extensive execution capabilities.

With our team, you can experience a collaborative environment with highly skilled multidisciplinary experts, where everyone has room to build bigger and bolder ideas. Being part of Metyis means you can speak your mind and be creative with your knowledge. Imagine the things you can achieve with a team that encourages you to be the best version of yourself.

We are Metyis. Partners for Impact.

What we offer

• Interact with senior stakeholders on regular basis, to drive their business towards impactful change.
• Become the go-to person for end-to-end data handling, management and analytics processes.
• Work with Data Scientists to take data throughout its lifecycle - acquisition, exploration, data cleaning, integration, analysis, interpretation and visualization.
• Become part of a fast-growing international and diverse team

What you will do

• Design, build and maintain big data architecture and data pipeline.
• Define and implement software solutions for scalability and maintainability.
• Solve problems, disaggregate issues, develop hypotheses and develop actionable solutions to support data engineering and data scientists teams.
• Kept up to date with latest industry knowledge and news to maintain and improve enterprise and business platforms.
• Develop and implement CI/CD pipeline automation solutions.
• Support and improve agile culture in your team.
• Collaborate and work closely with teams to better understand the end-user requirements.

What you’ll bring

• 5+ years of experience in a similar role with experience in developing and deploying ETL solutions on Azure.
• Solid knowledge of data warehousing principles, concepts and best practices (e.g. ODS, data marts, data lakes, data vault, 3NF)
• Good understanding of common platforms and practices regarding digital development; Cloud based setups, Advanced Analytics and computing environments and GitHub (or similar) and CI/CD workflows.
• Experience in setting up automated testing frameworks including unit tests, integration tests.
• Understanding of modern cloud based architecture (Lambda & Kappa architectures).
• Advanced SQL, data transformation and data profiling skills
• Experience of building production ETL/ELT pipelines at scale
• Data governance experience
• 3-5 years of hands on experience with Azure: Data factory, Databricks, Synapse (DWH), Azure Functions, App logic, and other data analytics services, including streaming;
• Experience with Airflow, Kubernetes, Great expectations is a plus
• Programming languages: Python (pyspark), scripting languages like bash
• Basic knowledge of PowerBI is a plus
• Customer centric, passionate about delivering great digital products and services
• Demonstrating true engineering craftsmanship mindset
• Passionate about continuous improvement and working collaboratively
• Strong problem-solving skills coupled with the ability to convey designs and ideas to a wider audience
• Comfortable with ambiguity, highly autonomous
• A Bachelor's Degree in Computer Science, Mathematics, Economics, Engineering, Operations Research, Statistics, Business or other related technical disciplines (Master's Degree is a plus);