Senior Data Engineer

Key Skills

Apache Airflow, Apache Flink, Apache Kafka, AWS, Big Data Technology, Data Analytics, Data Engineering, Data science techniques, Effective communication skills, ETL/ELT technologies, NoSQL, Python Programming, SPARK Programming, SQL

Job Description

We are seeking a skilled Data Engineer to join our Micro-Mobility division. As a Data Engineer, you will play a key role in designing and implementing robust data pipelines in AWS, utilizing technologies like EMR, Hudi, Kafka. Your responsibilities will include integrating data from diverse sources, with a future focus on MongoDB.

Your Responsibilities

• Build data pipelines that clean, transform, and aggregate data from disparate sources
• Experience in building or maintaining ETL/ELT processes
• Implements processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it
• Performs data analysis required to troubleshoot data related issues and assist in the resolution of data issues
• Use agile software development processes to make iterative improvements
• Strong communication skills and ability to collaborate with data science, analytics and other consumer teams

Who we are looking for

• Experience with Python, SQL, No-SQL and data visualization/exploration tools
• Experience with distributed systems utilizing tools such as Spark and Kafka
• Familiarity with the AWS ecosystem and analytics services
• Ability to work on a dynamic, research-oriented team that has concurrent projects

Your preferred qualifications

• Experience with containerization technologies such as Docker and Kubernetes
• Knowledge of stream processing frameworks like Apache Flink or Apache Storm
• Experience with workflow orchestration tools such as Apache Airflow or Luigi
• Certifications in relevant technologies (e.g., AWS Certified Big Data - Specialty)

What we offer

At Magna, you can expect an engaging and dynamic environment where you can help to develop industry-leading automotive technologies. We invest in our employees, providing them with the support and resources they need to succeed. As a member of our global team, you can expect exciting, varied responsibilities as well as a wide range of development prospects. Because we believe that your career path should be as unique as you are.

Site Benefits

Details of our benefits package will be provided during the recruitment process.