Data Engineer

Key Skills

Amazon RedShift, Amazon Simple Storage Service (S3), Analytical and Problem solving, AWS, AWS Glue, Big Data Technology, Cloud computing, Data Biuld Tool - dbt, Data Governance, Data Modeling, Data pipelines, Data Privacy, Data Warehousing, Databricks, DevOps, Effective communication skills, ETL frameworks, Lambda, Optimization, Spark Core, SQL

Job Description

We are in search for a veteran Data Engineer to join our team. As a Data Engineer, you will play a crucial role in designing, implementing, and maintaining our data infrastructure and pipelines. You will collaborate with cross-functional teams, including machine learning engineers, data scientists, and devops, to ensure the availability, reliability, and scalability of our data infrastructure.

Key Responsibilities:

Design, develop, and maintain scalable data pipelines and ELT processes to extract, load, and transform data from various sources into our data warehouse.

Optimize and tune data pipelines to ensure efficient data processing and improve overall system performance.

Work closely with cross-functional teams to identify and address data quality and integrity issues, ensuring data accuracy and consistency.

Implement and maintain data governance policies, ensuring compliance with data privacy and security regulations.

Monitor data pipelines and perform timely troubleshooting and issue resolution to minimize downtime and maintain data availability.

Stay up to date with the latest technologies and best practices in data engineering, recommending and implementing improvements to existing systems and processes.

Collaborate with DevOps and infrastructure teams to deploy and manage data infrastructure on AWS.

Create and maintain comprehensive documentation of data pipelines, processes, and systems.

Required Skills & Experience: Extensive experience with SQL, Spark, Lakehouse, Databricks, and dbt.

Strong understanding of data modeling concepts and experience with designing efficient data schemas.

Proficient in working with large-scale data processing frameworks and tools.

Familiarity with relevant AWS services, such as RDS, Lambda, S3, Glue, and Redshift.

Experience in building and optimizing ELT processes and data pipelines.

Solid understanding of data warehousing concepts and best practices.

Knowledge of data governance and data privacy regulations.

Strong problem-solving skills and the ability to troubleshoot and resolve complex data issues.

Excellent communication and collaboration skills to work effectively with cross-functional teams.

Strong attention to detail and a commitment to delivering high-quality work.

Keen attention to what's happening in the big data world and be able to drive innovation to significantly reduce backend/frontend operational overhead.

Predelo is rapidly growing and solving some of the previously unsolved data challenges. Apply today and join our dynamic team!