Lead Data Engineer

Key Skills

AWS, Azure, clickHouse, Database, Design, OLAP, Python Programming

Job Description

Intuition Machines uses AI/ML to build enterprise security products. We apply our research to systems that serve hundreds of millions of people, with a team distributed around the world.

If you enjoy working at scale on both architecture and data, engineering our backend systems may be your ideal job. Our approach is simple: light specs, small teams, and rapid iteration.

As a Lead Data Engineer you will be responsible for technical leadership of data engineering projects. You will do that by designing and improving high throughput data pipelines, promoting best practices in terms of high performance data processing, infrastructure setup and development process. You will have the ability to shape the data engineering capabilities for state of the art, large scale security and machine learning products.

What will you do:

• Lead the data engineering initiatives and projects
• Design, document and build scalable data infrastructure
• Collaborate with software engineers, ML engineers, product managers and growth teams
• Set up data quality, monitoring and alerting infrastructure
• Ensure high performance and availability of our data infrastructure

What we are looking for:

• Proven experience in designing and implementing end to end data solutions (+5 years of experience)
• Strong python programming skills with emphasis on clean, readable and testable code
• Experience with high throughput data systems and streaming architectures
• Experience working with Kafka infrastructure and applications
• Solid understanding of OLAP databases (preferably Clickhouse)
• Hands on experience with Kubernetes
• Familiarity with public cloud providers (AWS or Azure)

Nice to haves:

• Familiarity with security frameworks, attack vectors, botnets
• Experience working with IaC and GitOps solutions
• Experience with monitoring, observability and data quality tools
• Experience with exploratory data analysis and data science solutions

What do we offer:

• Fully remote position with flexible working hours #Li-remote.
• An inspiring team of colleagues spread all over the world.
• Pleasant, modern development and deployment workflows: ship early, ship often.
• High impact: lots of users, happy customers, high growth, and cutting edge R&D.
• Flat organization, direct interaction with customer teams.