Director, Data Engineering

Key Skills

Java Programming, SQL, Agile methodologies, AWS, Apache Kafka, Scala Programming, Data science techniques, NoSQL, Apache Hive, HDFS, YARN, SPARK Programming, Streaming APIs, KSql, Large scale data processing, ETL frameworks, Dimensional Modelling

Job Description

Our Tech Team is the core of Dream11’s mobile-first cross-platform (Android & iOS, Mobile + Desktop PWA) product, serving more than 10 Crore users with over 70 million rpm (requests per minute) at peak with user concurrency of 5.5 million. Our tech stack is hosted on AWS and comprises multiple distributed systems like Cassandra, Aerospike, Akka, Voltdb, Ignite etc.

We have around 100+ micro-services primarily written in Java backed by vert.x framework. They serve isolated product features with discrete architectures to serve the respective use-cases. We have a completely in-house data infrastructure built on top of Kafka, Redshift, Spark, Druid etc. which powers our Machine Learning and Predictive Analytics use-cases. We ingress Terabytes of Data every day, which flows all over our Data pipelines to power a plethora of use-cases.

Your Role:

• Building and maintaining self-service Data Platform for Dream Sports
• Reviewing existing implementations and propose cost effective solutions
• Interacting with business stake holders and proposing/implementing solutions to solve business problems
• Working with like minded team of Data Engineers with expertise on working at large scale
• Building scalable data processing platform and frameworks for serving Analytics and Data Science use cases
• Building and managing a high performing team and supporting them through executing company's objectives

Must Have:

• 8+ years of experience in Dimensional Modelling and large scale Data Warehouse implementations
• Deep expertise in building real-time and batch analytics ETL solutions using big data technologies (Spark/ Kafka/KSql/Yarn/Hive/HDFS/Streaming etc) and NoSQL Stores
• Expert level programming experience in Java or Scala
• Strong SQL background, cloud based experience (AWS preferred) and proficient with scripting languages - python, shell etc
• Proven experience in managing stakeholders, mentoring teams and driving data platform projects

Good to Have:

• Have worked previously in a high-growth, fast paced tech startup
• Prior working experience in agile environment

Apply Online at Dream Sports