Lead Machine Learning Engineer, Peacock

Key Skills

Apache Airflow, Apache Flink, Apache Kafka, Big Data Technology, Data pipelines, Docker, Google BigQuery, Google Cloud Platform (GCP), Kubernetes-K8s, Leadership Skill, Machine learning techniques, Python Programming, Snowflake, SPARK Programming, SQL

Job Description

As part of the Direct-to-Consumer Decision Sciences team, the Lead ML Engineer will be responsible for creating a connected data ecosystem that unleashes the power of our streaming data. We gather data from across all customer/prospect journeys in near real-time, to allow fast feedback loops across territories; combined with our strategic data platform, this data ecosystem is at the core of being able to make intelligent customer and business decisions.

In this role, the Lead ML Engineer will share responsibilities in the development and maintenance of an optimized and highly available machine learning pipelines that facilitate decision making by the business, as well as support ongoing operations related to the Direct to Consumer data ecosystem

Responsibilities include, but are not limited to:

• Design and participate in the development, testing, scaling and maintenance of data pipelines from a variety of source systems and streams (Internal, third party, cloud based, etc.), according to business and technical requirements.
• Deliver observable, reliable and secure software, embracing a “you build it you run it” mentality, and focusing on automation and GitOps.
• Continually work on improving the codebase and have active participation and oversight in all aspects of the team, including agile ceremonies.
• Take an active role in story definition, assisting business stakeholders with acceptance criteria, clarifying requirements with the team, and supporting execution and delivery.
• Develop and champion best practices, striving towards excellence and raising the bar within the department.
• Develop solutions combining data blending, profiling, mining, statistical analysis, and machine learning, to better define and curate models, test hypothesis, and deliver key insights
• Operationalize data processing systems (dev ops)
• In cooperation w/ the data science team, productize machine learning models using workflow orchestrators such as Kubeflow/VertexAI or Apache Airflow
• Develop data catalogs and data cleanliness to ensure clarity and correctness of key business metrics
• Create and deliver effective presentations on a variety of subjects both internally to the team, and externally to the business and stakeholders
• Collaborate with Principal Engineers and Architects to share and contribute to the broader technical vision, and coordinate with partner teams to define requirements and drive the expansion of capabilities across the organization.

Qualifications

• Experience with near Real Time & Batch Data Pipeline development in a similar Big Data Engineering role.
• Experience designing, developing, and optimizing enterprise data processing applications utilizing Python and SQL
• Experience implementing scalable, distributed, and highly available systems in cloud platforms (GCP preferred)
• Demonstrated proficiency with the following (or similar) technologies:
• Kubernetes, Docker
• Apache Beam, Apache Flink, Apache Spark
• Google BigQuery, Snowflake
• Google BigTable
• Google Pub/Sub, Kafka
• Apache Airflow, Kubeflow
• Experience in processing structured and unstructured data into a form suitable for analysis and reporting with integration with a variety of data metric providers ranging from advertising, web analytics, and consumer devices.
• Experience operationalizing data science models/products
• Bachelors’ degree with a specialization in Computer Science, Engineering, Physics, other quantitative field or equivalent industry experience.

Desired Characteristics

• Ability to work effectively across functions, disciplines, and levels
• Team-oriented and collaborative approach with a demonstrated aptitude, enthusiasm and willingness to learn new methods, tools, practices and skills
• Ability to recognize discordant views and take part in constructive dialogue to resolve them
• Pride and ownership in your work and confident representation of your team to other parts of NBCUniversal.
• Strong Test-Driven Development background, with understanding of levels of testing required to continuously deliver value to production.