Data Engineer (Airflow) (Freelancer)

Key Skills

Amazon Elastic Compute Cloud-EC2, Amazon RedShift, Amazon Simple Storage Service (S3), Analytical and Problem solving, AWS, Bash scripting, DevOps, Python Programming, Terraform

Job Description

Mactores is a trusted leader among businesses in providing modern data platform solutions. Since 2008, Mactores have been enabling businesses to accelerate their value through automation by providing End-to-End Data Solutions that are automated, agile, and secure. We collaborate with customers to strategize, navigate, and accelerate an ideal path forward with a digital transformation via assessments, migration, or modernization.

We are seeking a talented and experienced Airflow Administrator to join our team. In this role, you will be responsible for migrating Apache Airflow workflows from on-premises infrastructure to Amazon Managed Workflows for Apache Airflow (MWAA) on the AWS cloud.

The successful candidate will have a strong background in Apache Airflow, and AWS services, and experience in migration projects. If you are passionate about cloud technologies and have a proven track record of successful migrations, we want to hear from you.

What you will do?

• Evaluate existing on-premises Apache Airflow workflows to identify dependencies, scheduling requirements, and integration points
• Collaborate with stakeholders to understand business requirements and constraints for the migration
• Design and implement migration strategies to move Airflow workflows to AWS MWAA while minimizing downtime and ensuring data integrity
• Configure and optimize Airflow DAGs for AWS MWAA compatibility
• Implement security best practices during the migration process
• Set up and configure AWS MWAA environments in accordance with best practices
• Integrate AWS services such as S3, Redshift, and others as needed for Airflow workflows
• Establish monitoring and logging mechanisms to ensure the health and performance of migrated workflows
• Conduct testing and validation to verify the correctness of the migration
• Create comprehensive documentation of migration processes, configurations, and best practices
• Provide training and knowledge transfer to internal teams for managing AWS MWAA
• Provide ongoing support and troubleshooting assistance for any issues that arise after migration
• Perform performance tuning and optimization of Airflow workflows on AWS MWAA

What are we looking for?

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• Proven experience in migrating Apache Airflow workflows to AWS MWAA or other cloud platforms.
• Strong expertise in Apache Airflow, including DAG design and task scheduling.
• AWS certification(s) in relevant areas (e.g., AWS Certified Data Analytics Specialty, AWS Certified DevOps Engineer) is a plus.
• Familiarity with AWS services such as S3, Redshift, EC2, IAM, and CloudWatch.
• Knowledge of infrastructure as code (IaC) tools like Terraform or AWS CloudFormation.
• Proficient in scripting languages (e.g., Python, Bash) for automation.
• Excellent problem-solving skills and attention to detail.
• Strong communication and collaboration skills to work effectively with cross-functional teams.
• Ability to work independently and handle multiple tasks simultaneously.

Life at Mactores

We care about creating a culture that makes a real difference in the lives of every Mactorian. Our 10 Core Leadership Principles that honor Decision-making, Leadership, Collaboration, and Curiosity drive how we work.

• Be one step ahead
• Deliver the best
• Be bold
• Pay attention to the detail
• Enjoy the challenge
• Be curious and take action
• Take leadership
• Own it
• Deliver value
• Be collaborative

We would like you to read more details about the work culture on https://mactores.com/careers

The Path to Joining the Mactores Team

At Mactores, our recruitment process is structured around three distinct stages:

Pre-Employment Assessment:

You will be invited to participate in a series of pre-employment evaluations to assess your technical proficiency and suitability for the role.

Managerial Interview: The hiring manager will engage with you in multiple discussions, lasting anywhere from 30 minutes to an hour, to assess your technical skills, hands-on experience, leadership potential, and communication abilities.

HR Discussion: During this 30-minute session, you'll have the opportunity to discuss the offer and next steps with a member of the HR team.

At Mactores, we are committed to providing equal opportunities in all of our employment practices, and we do not discriminate based on race, religion, gender, national origin, age, disability, marital status, military status, genetic information, or any other category protected by federal, state, and local laws. This policy extends to all aspects of the employment relationship, including recruitment, compensation, promotions, transfers, disciplinary action, layoff, training, and social and recreational programs. All employment decisions will be made in compliance with these principles.