Senior Data Engineer

Key Skills

Analytical and Problem solving, AWS, Data Engineering, Data pipelines, Data science techniques, Effective communication skills, ETL frameworks, Infrastructure as code, Large data sets, Optimization, Python Programming, SQL, Tableau, Writing

Job Description

About The Role

As a Senior Data Engineer, you will be responsible for the end-to-end data pipeline, ensuring the reliability, efficiency, and scalability of our data systems. You will collaborate closely with cross-functional teams, including data scientists, analysts, and software engineers, to develop robust data solutions that empower data-driven decision-making.

What You'll Do

• Create optimal data pipeline architecture
• Develop, and maintain end-to-end scalable data infrastructure and pipelines.
• Identify, design, and implement process improvements, automating manual processes and optimizing data delivery.
• Assist internal stakeholders with data-related technical issues and support their data infrastructure needs.
• Develop data tools for analytics and data scientists, contributing to product innovation.
• Collaborate with data and analytics experts to enhance functionality in data systems.
• Design and implement data models, ensuring integrity and consistency.
• Identify and resolve performance bottlenecks, optimizing queries and processing.
• Implement data governance best practices for quality, security, and compliance.
• Stay informed about emerging technologies, contribute to tool selection, and enhance data infrastructure.
• Create and maintain comprehensive documentation for data engineering processes and workflows.
• Drive the team's data strategy, technical roadmap, and data storage solutions.
• Empower the team for self-servicing and efficient troubleshooting.

Your Background

• Bachelor’s degree or equivalent experience.
• 7+ years of experience in Software product engineering
• Extensive experience in using SQL in an advanced capacity in query authoring, tuning, and identifying useful abstractions
• 5+ years of hands-on experience in Python, demonstrating proficiency in data wrangling/ object-oriented programming
• Expertise in designing, creating, managing, and utilizing large datasets.
• Ability to build efficient, flexible, extensible, and scalable ETL and reporting solutions.
• Root cause analysis experience on internal and external data and processes.
• Cloud-based database platforms experience, preferably Snowflake.
• 3+ years of git experience for version control and collaboration.
• 3+ years of experience working with AWS cloud technology.
• 3+ years of experience working with workflow management platforms like airflow/dagster
• 2+ years experience using Tableau to build impactful reports and dashboards
• Experience working with dbt is a plus
• Strong analytical and problem-solving skills.
• Cross-functional team collaboration experience in a dynamic environment.
• Proven track record of navigating ambiguity, prioritizing needs, and solving impactful business problems.
• Excellent written and verbal communication skills for technical and non-technical audiences.
• Empathy-driven, supporting team success.
• Remote work experience with a U.S.-based team is preferred.