Senior Data Platform Engineer

Key Skills

Amazon Athena, Amazon RedShift, Apache Airflow, Apache Hadoop, Apache Kafka, Automation, AWS Kinesis, Azure, Big Data Technology, C++, Cloud computing, Data pipelines, Data Warehousing, Docker, Effective communication skills, Google BigQuery, Google Cloud Platform (GCP), Kubernetes-K8s, MongoDB, NoSQL, Presto, Python Programming, RabbitMQ, RDBMS, React.js, Snowflake, Spark Core, Terraform

Job Description

Luminar is a global automotive technology company ushering in a new era of vehicle safety and autonomy. For the past decade, Luminar has built an advanced hardware and software/AI platform to enable its more than 50 industry partners, including the majority of global automotive OEMs. From consumer vehicle programs with Volvo Cars and Mercedes-Benz to technology partnerships including NVIDIA and Mobileye, Luminar is poised to be the first automotive technology company to enable next-generation safety and autonomous capabilities for global production vehicles.

Team Overview:

The Data team builds the services and infrastructure to leverage data from our vehicles and operations. We are building the next generation data platform as we scale our offerings. You will develop and implement the systems that power Luminar’s data-driven improvement and growth, and save lives by doing it.

Responsibilities

• Development of large scale data infrastructure and services. 
• Designing, building, and maintaining the infrastructure that transforms data at scale into insights:
• Ingestion of real-world vehicle data.
• Automated labeling and data enrichment.
• Process, index and curate big data into value-driving datasets.
• Generation of ground truth information.
• Analysis of data quality.
• Partnering with engineering and ML teams to define data consumption patterns and establish best practices.
• Establishing robust data integrity and systems monitoring.

Minimum Qualifications

• 7+ years of relevant industry experience: 
• Building backend services and/or data pipelines at scale.
• Hands-on Big Data software and infrastructure development skills.
• Experience with NoSQL databases and/or Relational databases.
• Accomplished in Python.
• Demonstrate a high degree of software craftsmanship and the ability to own a complex system end to end.
• Strong communication skills, being able to collaborate with the wider software organization.
• Able to work successfully in cross-functional teams, especially across organizational and geographical boundaries.
• Incident management and root cause analysis.
• Performance Engineering.
• Previous experience in the Automotive / Robotics industry.

Preferred Qualifications

• Proficiency in C++.
• Experience with MongoDB.
• Experience with processing frameworks like Spark / Hadoop.
• Experience with pipeline / workflow frameworks: Argo Workflows or Airflow.
• Hands-on experience with cloud computing: AWS / GCP / Azure..
• Experience with automation of data infrastructure: Docker / Kubernetes / Terraform.
• Experience with SRE practices - running services reliably in production environments:
• Proficiency in front-end development, preferably using React.
• Experience with big data / data warehouse tools like Redshift, Presto/Athena, BigQuery and Snowflake.
• Experience with streaming and message queueing technologies: Kafka / Kinesis / RabbitMQ.
• Hands-on experience with data storage and transfer schemes like Avro, Parquet and Protobuf.
• Experience with ROS.