Lead Data Engineer

Key Skills

Java Programming, Python Programming, SQL, Apache Hadoop, Apache Kafka, Scala Programming, NoSQL, Snowflake, Apache Airflow, Google Cloud Platform (GCP), R Programming, SPARK Programming

Job Description

NBCUniversal owns and operates over 20 different businesses across 30 countries including a valuable portfolio of news and entertainment television networks, a premier motion picture company, significant television production operations, a leading television stations group, world-renowned theme parks and a premium ad-supported streaming service.

Here you can be your authentic self. As a company uniquely positioned to educate, entertain and empower through our platforms, Comcast NBCUniversal stands for including everyone. We strive to foster a diverse and inclusive culture where our employees feel supported, embraced and heard. We believe that our workforce should represent the communities we live in, so that together, we can continue to create and deliver content that reflects the current and ever-changing face of the world. Click here to learn more about Comcast NBCUniversal’s commitment and how we are making an impact.

Job Description

Welcome to Peacock, the dynamic new streaming service from NBCUniversal. Here you’ll find more than a job. You’ll find a fast-paced, high-flying team for unique birds that want to be at the epicenter of technology, sports, news, tv, movies and more. Our flock works hard to connect people to what they love, each other and the world around them by creating shared experiences through culture-defining entertainment.

As a company, we embrace the power of difference. Our team is committed to creating an organization that champions diversity and inclusivity for all by curating content and a workforce that represents the world around us. We continue to challenge ourselves and the industry by being customer-centric, data-driven creatures of innovation. At Peacock, we are determined to forge the next frontier of streaming through creativity, teamwork, and talent.

Here you can fly to new heights!

NBCUniversal, the global media company that brought you some of the world’s most iconic television and film franchises, including: The Tonight Show, Saturday Night Live, Keeping Up With The Kardashians, The Real Housewives, Mr. Robot, The Voice, This Is Us, The Fast & The Furious, Jurassic Park, Minions, and more - is launching an all-new direct-to-consumer streaming service. It will seamlessly bring together the breadth and depth of NBCU’s broadcast and cable television series, movie titles, premier sporting events, and renowned news reporting... all in one destination… all in one app. 

Introducing Peacock, NBCUniversal's new streaming service that combines timeless shows and movies, exclusive originals, kids programming and current hits, with timely news, sports and pop culture. All together. All in one app. 

As part of the Direct-to-Consumer Decision Sciences team, the Lead Data Engineer will be responsible for creating a connected data ecosystem that unleashes the power of our streaming data. We gather data from across all customer/prospect journeys in near real-time, to allow fast feedback loops across territories; combined with our strategic data platform, this data ecosystem is at the core of being able to make intelligent customer and business decisions.

In this role, the Lead Data Engineer will share responsibilities in the development and maintenance of an optimized and highly available data pipelines that facilitate deeper analysis and reporting by the business, as well as support ongoing operations related to the Direct to Consumer data ecosystem

Responsibilities include, but are not limited to:=

• Help manage a high-performance team of Data Engineers
• Provide guidance in the designing, building, testing, scaling, and maintaining of data pipelines from a variety of sources (Internal, third party, cloud based, etc.), according to business and technical requirements.
• Deliver observable, reliable and secure software, embracing “you build it you run it” mentality, and focus on automation and GitOps.
• Continually work on improving the codebase and have active participation and oversight in all aspects of the team, including agile ceremonies.
• Take an active role in story definition, assisting business stakeholders with acceptance criteria.
• Work with Principal Engineers and Architects to share and contribute to the broader technical vision.
• Develop and champion best practices to improve the efficiency of the team, striving towards excellence and raising the bar within the department.
• Operationalize data processing systems utilizing DevOps principles

Qualifications

• 5+ years relevant experience in Data Engineering
• Experience of near Real Time & Batch Data Pipeline development in a similar Big Data Engineering role.
• Programming skills an object oriented programming language such as Python, Java, or Scala and experience in writing reusable/efficient code to automate analysis and data processes
• Experience in processing structured and unstructured data into a form suitable for analysis and reporting with integration with a variety of data metric providers ranging from advertising, web analytics, and consumer devices
• Experience implementing scalable, distributed, and highly available systems using Google Cloud
• Hands on programming experience of the following (or similar) technologies: Apache Beam, Scio, Apache Spark, and Snowflake.
• Experience in progressive data application development, working in large scale/distributed SQL, NoSQL, and/or Hadoop environment.
• Develop data catalogs and data cleanliness to ensure clarity and correctness of key business metrics
• Experience building streaming data pipelines using Kafka, Spark, or Flink
• Data modelling experience (operationalizing data science models/products) a plus
• Bachelors’ degree with a specialization in Computer Science, Engineering, Physics, other quantitative field or equivalent industry experience.

• Must submit an attestation disclosing your COVID-19 vaccination status and, if partially or fully vaccinated, submitting your vaccination record no later than 7 days following commencement of employment.

  • Must be fully vaccinated against COVID-19 at the commencement of employment or adhere to enhanced protocols in select work settings or where jurisdictionally mandated.

  • Must be willing to adhere to all Company COVID-19 workplace safety policies and protocols.

  Desired Characteristics

• Experience with graph-based data workflows using Apache Airflow
• Experience building and deploying ML pipelines: training models, feature development, regression testing
• Strong Test-Driven Development background, with understanding of levels of testing required to continuously deliver value to production.
• Experience with large-scale video assets
• Ability to work effectively across functions, disciplines, and levels
• Team-oriented and collaborative approach with a demonstrated aptitude, enthusiasm and willingness to learn new methods, tools, practices and skills
• Ability to recognize discordant views and take part in constructive dialogue to resolve them

  • Pride and ownership in your work and confident representation of your team to other parts of NBCUniversal