Data Engineer II

Key Skills

Apache Flink, Apache Hadoop, Apache Kafka, AWS, Azure, Big Data Technology, Deep Learning, ETL frameworks, ETL/ELT technologies, Google Cloud Platform (GCP), Java Programming, Machine learning techniques, Python Programming, Scala Programming, Spark Core, SQL

Job Description

What You'll do:

• Responsible for defining the Source of Truth (SOT), Dataset design for multiple Uber teams.
• Identify unified data models collaborating with Data Science teams
• Streamline data processing of the original event sources and consolidate them in source of truth event logs
• Build and maintain real-time/batch data pipelines that can consolidate and clean up usage analytics
• Build systems that monitor data losses from different sources and improve the data quality
• Owns the data quality and reliability of the Tier-1 & Tier-2 datasets including maintaining their SLAs, TTL and consumption
• Devise strategies to consolidate and compensate the data losses by correlating different sources
• Solve challenging data problems with cutting-edge design and algorithms.

What you'll need:

• 4+ years of extensive Data engineering experience working with large data volumes and different sources of data.
• Strong data modeling skills, domain knowledge, and domain mapping experience.
• Strong experience in using SQL language and writing complex queries.
• Experience with using other programming languages like Java, Scala, Python
• Good problem-solving and analytical skills
• Good communication, mentoring, and collaboration skills.
• Extensive experience in data engineering and working with Big data
• Experience with ETL or Streaming data and one or more of, Kafka, HDFS, Apache Spark, Apache Flink, Hadoop
• Experience backend services and familiarity with one of the cloud platforms ( AWS/ Azure / Google Cloud)