Staff Software Engineer, Spark/Data Processing

Key Skills

Big Data Technology, Database, Java Programming, Large scale data processing, NoSQL, PySpark, Python Programming, Software Development, Spark Core, SQL

Job Description

At Cruise, the Machine Learning Infrastructure organization (MLI) builds the essential tools, frameworks, and services used by every engineer at Cruise. MLI’s goal is to greatly accelerate the development cycle of AV engineers by building the infrastructure to process, analyze, and learn from the many petabytes of data coming from our AVs, simulated tests, and real time services, empowering engineers to focus on improving the car’s safety and performance.

As an engineer on this team, you will be responsible for building and supporting a petabyte-scale data platform in the cloud and providing powerful foundations for MLI’s tools, frameworks, and services. Responsibilities include ensuring scalable, transparent, and reliable data ingestion and management; development of fast, robust, and spike-resistant data consumption, data mining, and processing tools for the entire company; and development of orchestration for large-scale post-processing, and computational pipelines.

What you’ll be doing: 

• Use the latest cloud technologies to own, design, implement, and test scalable distributed data systems in the cloud. Champion engineering excellence by continuously improving systems and processes
• Develop, optimize and productionize the next generation data processing platform using Spark on cloud. Build self-serve capabilities to help customers to adopt the next generation data processing platform 
• Own technical projects from start to finish, contribute to the team’s product roadmap, and be responsible for major technical decisions and tradeoffs. Effectively participate in team’s planning, code reviews and design discussions
• Consider the effects of projects across multiple teams and proactively manage conflicts. Work together with partner teams and orgs to achieve cross-departmental goals and satisfy broad requirements
• Conduct technical interviews with well-calibrated standards and play an essential role in recruiting activities. Effectively onboard and mentor junior engineers and/or interns

What you must have: 

• 10+ years working with big data
• Experience building scalable infrastructure on the cloud with Python or Java (or similar)
• Experience working with relational and NoSQL databases
• Experience with building data processing system using Spark and its ecosystems
• Experience with Kafka, Hadoop, Beam or other data processing tools
• BS, MS or Ph.D. in Computer Science, Electrical Engineering, Mathematics, Physics, or another relevant field; or equivalent real-world experience
• Passionate about self-driving technology and its potential impact on the world
• Attention to detail and a passion for truth
• A track record of efficiently solving complex problems
• Startup mentality - openness to dealing with unknown unknowns and wearing many hats

We're Cruise, a self-driving service designed for the cities we love.

We’re building the world’s most advanced self-driving vehicles to safely connect people to the places, things, and experiences they care about. We believe self-driving vehicles will help save lives, reshape cities, give back time in transit, and restore freedom of movement for many.

In our cars, you’re free to be yourself. It’s the same here at Cruise. We’re creating a culture that values the experiences and contributions of all of the unique individuals who collectively make up Cruise, so that every employee can do their best work. 

Cruise is committed to building a diverse, equitable, and inclusive environment, both in our workplace and in our products. If you are looking to play a part in making a positive impact in the world by advancing the revolutionary work of self-driving cars, come join us. Even if you might not meet every requirement, we strongly encourage you to apply. You might just be the right candidate for us.

We're Cruise, a self-driving service designed for the cities we love.

We’re building the world’s most advanced self-driving vehicles to safely connect people to the places, things, and experiences they care about. We believe self-driving vehicles will help save lives, reshape cities, give back time in transit, and restore freedom of movement for many.

In our cars, you’re free to be yourself. It’s the same here at Cruise. We’re creating a culture that values the experiences and contributions of all of the unique individuals who collectively make up Cruise, so that every employee can do their best work. 

Cruise is committed to building a diverse, equitable, and inclusive environment, both in our workplace and in our products. If you are looking to play a part in making a positive impact in the world by advancing the revolutionary work of self-driving cars, come join us. Even if you might not meet every requirement, we strongly encourage you to apply. You might just be the right candidate for us.