Senior Data Engineer

Key Skills

Apache Airflow, Apache Hadoop, Apache Kafka, AWS, Big Data Technology, Data Architecture, Data Biuld Tool - dbt, Data Engineering, Data pipelines, Docker, Elasticsearch, Kubernetes-K8s, MapReduce, MongoDB, Postgres, Python Programming, REDIS, Scala Programming, SPARK Programming

Job Description

Our Senior Data Engineer will play an essential role by building the underlying infrastructures, collecting, storing, processing and analyzing large sets of data, while collaborating with researchers, architects, and engineers, in order to design and build high-quality data processing for our flows.

In this role, you are responsible for end-to-end development of the data pipeline and data models, working with major data flow that includes structured and unstructured data. You will also hold responsibility for operating parts of our production system. Your focus will be on developing and integrating systems that retrieve and analyzing data that influence people's lives. This role is based in the New York City Metropolitan Area and is a hybrid role working at least two days per week in our NYC office.

Our Technologies stack: Python, Spark, Airflow, DBT, Kafka, AWS, Docker, Kubernetes, MongoDB, Redis, Postgres, Elasticsearch, and more.

The ideal candidate will be:

• A technology enthusiast – who loves data and get shiver excitement from tech innovations
• Desire to know how things work and a greater desire to improve them
• Intellectual curiosity to find unusual ways to solve problems
• Comfortable taking on challenges and learning new technologies
• Comfortable working in a fast-paced dynamic environment

Qualifications:

• Bachelor’s Degree in Computer Science or related field
• 7+ years working experience of designing and implementing server-side Data solutions
• Proven experience in creating and optimizing big data processes, pipelines and architectures
• Proven experience with AWS ecosystem
• Proven experience with Kubernetes in Production
• Experience with at least one high-level system language (e.g. Python, Scala)
• Experience working on production grade projects
• Familiarity with Python
• Familiarity with Apache Airflow - Advantage
• Familiarity with distributed systems and technologies (Spark, Hadoop, MapReduce) - Advantage

The US base salary range for this full-time position is $133,224 to $199,836. Salary ranges are determined by role and level. Compensation is determined by additional factors, including job-related skills, experience, and relevant education or training. Please note that the compensation details listed in US role postings reflect the annual salary only, and do not include equity or benefits.