Lead Data Engineer, Data Platform

Key Skills

Java Programming, Python Programming, C++, C Programming, SQL, Cloud computing, Scala Programming, Machine learning techniques, Data science techniques, MATLAB Programming, PyTorch, TensorFlow, R Programming

Job Description

The Tubi Data Platform team is looking for a hands-on leader! As Tubi continues to grow, an insatiable appetite for data grows with it. Originally built to support our core ML models and recommendation engines, Tubi's data platform has continued to grow in use and scale but is starting to show its age. More real-time streaming use cases with many more dataset publishers and consumers require more innovation and leadership for our small data platform team. You will be responsible for scaling and maintaining high quality infrastructure to make sure we can support the business with performant, scalable, and accurate datasets. Leading the Data Platform team you will have an opportunity to have an outsize impact on an organization where data is core to everything we do. We would prefer this candidate to be located in the pacific standard time zone.

Responsibilities:

• Build and scale Tubi's primary Data Platform. You will start with a team of at least 5 direct reports
• Your team is responsible for all things Data Platform, from data ingestion pipelines, to the storage, delivery, and discoverability of high quality datasets
• You will build and maintain infrastructure to support real-time streaming pipelines that create massive datasets for use in Tubi's ML pipelines and Data Lakehouse
• You will manage infrastructure for both batch and streaming pipelines for 3rd party data integrations
• You will guide Data Governance with a tireless commitment to high data quality and usability
• You will be expected to be a hands-on leader, guiding your team by example. You will be expected to be technical and competent in the day-to-day data platform operations, but still big-picture enough to build a well-oiled, self-sustaining team that solves business needs

Your Background:

• Master’s or Bachelor’s degree from an accredited university, or equivalent work experience 
• 7+ years hands on industry experience, and 2+ years of experience leading and managing a team
• Experience being a team lead & comfortable managing a growing and proactive team
• Hands-on experience building out PB scale data lakes on transactional data formats (Delta, Iceberg)
• Deep experience with publishing and consuming streaming datasets with either Apache Spark (Structured Streaming) or Apache Flink
• Strong understanding of distributed queues (Kafka/Kinesis) and distributed databases (Cassandra, Redis)
• Excellent communication / coordination skills to work with a cross functional team to onboard new datasets for diverse business verticals
• Experience working with Terraform, k8s, and building CI/CD pipelines for data applications
• Strong experience in a Scala or Java (scala preferred)