Head of Infrastructure Data Science

Key Skills

Apache Hadoop, C Programming, C++, Data science techniques, Java Programming, Machine learning techniques, MapReduce, Python Programming, PyTorch, R Programming, Scala Programming, SQL, TensorFlow

Job Description

At Databricks, we are obsessed with enabling data teams to solve the world's toughest problems, from security threat detection to cancer drug development. We do this by building and running the world's best data and AI infrastructure platform, so our customers can focus on the high-value challenges that are central to their missions.

Founded in 2013 by the original creators of Apache Spark, Databricks has grown from a tiny corner office in Berkeley, CA to a global organization with over 1500 employees. Thousands of organizations, from small to Fortune 100, trust Databricks with their mission-critical workloads, making us one of the fastest-growing SaaS companies in the world.

Our engineering teams build highly technical products that fulfill real, important needs in the world. We constantly push the boundaries of data and AI technology, while simultaneously operating with the resilience, security, and scale that is critical to making customers successful on our platform.

Do you dream about making infrastructure more efficient with data science? Databricks’s scale and range of initiatives present a tremendous opportunity for data science to tackle interesting, important challenges around optimizing our infrastructure. How do we better understand and forecast cloud computing costs and the input variables that drive costs? How can we better evaluate performance of our service? How do we better detect regressions hidden among billions of time series data points in our monitoring infrastructure? Can we detect server-side performance regressions in a high velocity engineering environment where there are strong trend and seasonality effects?

We’re looking for a leader to start a new Infrastructure Data Science group. The vision for the Infrastructure Data Science group is to use statistical and machine learning techniques to optimize the operations--and enable the continued growth of--Databricks’s infrastructure. It’s a group of “full stack” data scientists that partners with teams supporting all of Databricks’s product and infrastructure, focusing on long-term strategic initiatives that make Databricks infrastructure more efficient, reliable, and scalable.

The impact you will have:

• Identify how data science can be applied to improve, optimize, and expand Databricks’s infrastructure across a variety of domains, with emphasis on long-term and strategic initiatives
• Create data awareness and drive behavioral change to make significant efficiency improvements across the engineering organization
• Partner with engineering operations, performance engineering, infra engineering, and other teams to develop KPIs, dashboards, and robust models for more thorough understanding of infrastructure performance, SLAs, and service reliability
• Maintain responsibility for ensuring that our margin is well understood, identifying and driving optimization activities to ensure we are making the most out of our investments. Create reporting, monitoring, and forecasting to ensure awareness of cost trending and detection of cost anomalies
• Build anomaly detection models to monitor the service health and data quality, and alert any significant issues with insights of drivers and potential root causes of the issues. 
• Represent the data science discipline throughout the organization, having a powerful voice to make us more data-driven

What we look for:

• 8+ years of data science and advanced analytics experience in high velocity, high-growth companies
• 5+ years of management experience hiring and developing teams
• 2+ years of experience working with cloud products (AWS, GCP, or Azure)
• Experience creating cost optimization and anomaly detection models with engineering teams that utilize cloud services a big plus
• Strong software engineering background is a plus
• Experience developing analytics and data science capabilities in a cloud environment
• Knowledge of statistics and rigorous analytical techniques, experience with data visualization tools
• Leadership skills and experience to lead across functional and organizational lines
• Strong communication skills to explain and evangelize analytics and data science to other engineers and executives
• Bias to action and passion for delivering high-quality data solutions
• MS or Ph.D. in quantitative fields (CS, Statistics, Math, or Engineering)

Benefits

• Private medical insurance
• Accident coverage
• Employee's Provident Fund
• Equity awards
• Paid parental leave
• Gym reimbursement
• Annual personal development fund
• Work headphones reimbursement
• Business travel insurance