Big Data Engineer (Python)

Key Skills

Amazon Athena, Amazon Simple Storage Service (S3), Apache Airflow, Apache Hadoop, Apache Kafka, AWS, AWS Glue, Azure, Big Data Technology, Continuous Integration & Continuous Delivery - CI/CD, Data Architecture, Data Engineering, Data pipelines, Data science techniques, Docker, Grafana, Jenkins, Kubernetes-K8s, PySpark, Python Programming, SQL, Tableau

Job Description

This squad will be working as a unity and would follow a project-based approach. The goal of the Data Solutions Tribe is to accelerate the onboarding of domains into our new Data Platform/Data Science Platform.

The role

As a Staff Data Engineer, you will be responsible for spreading best data engineering practices within the organization and in the team. You will support us in designing, building, and maintaining our data infrastructure, with special focus on flexibility and scalability. You will have the opportunity to work on challenging data-related problems and to have a great impact on the organization.

What you’ll do

• Supporting us with moving to a data mesh architecture
• Defining standard working methods and processes to be respected by all chapter members, e.g., best practices in developing data pipelines, data architectures, testing, and deploying changes, documentation standards
• Enabling quick and accurate decision-making within the teams by facilitating discussions
• Creating and coordinating communities
• Coaching and mentoring data engineers in order to support their growth
• Setting and spreading best practices for Data Engineering across the organization
• Providing solutions for complex business problems, enabling insights that can empower better decision-making
• Collaborating with our Senior Director of Data on strategic projects

What you’ll bring

• You are a strong data engineer having worked with Kafka, the Hadoop Ecosystem (Hadoop, Kafka, Spark), with cloud based services (AWS S3, EMR, Glue) and with containers (Docker, Kubernetes, ECS, EKS).
• You bring hands-on experience in creating and managing an enterprise-scale Data Platform, while setting best practices for security, privacy, monitoring & alerting, and CI/CD (ideally, e.g., Azure, Prometheus, AlertManager, Grafana, Jenkins, Concourse, Airflow, Tableau, Alation)
• You have coached and mentored team members in previous roles
• You leverage data to prioritize your work and to influence others
• You have managed senior stakeholders in complex organizations and in changing environments
• You are willing to implement change and improvements when seeing an opportunity

Tech stack: Python, AWS, PySpark, SQL, Kafka, AirFlow.

About us: We are a global technology company that offers data and solutions to establish a safer, more effective advertising ecosystem. We partner with advertisers and publishers to protect their investments, capture consumer attention, and drive business impact. Founded in 2009, is headquartered in New York with global operations in 13 countries. Our growth and innovation have been recognized in the Inc. 5000, Crain’s Fast 50, Forbes America’s Most Promising Companies, and I-COM’s Smart Data Marketing Technology Company.