Senior Software Engineer, Data Engineering

Key Skills

Java Programming, Design, Python Programming, Database, SQL, Apache Hadoop, Scala Programming, NoSQL, Apache Hive, Snowflake, Apache Airflow, Continuous Integration & Continuous Delivery - CI/CD, Amazon RedShift, SPARK Programming, Data Engineering, Data pipelines

Job Description

Our team builds the enabling technology behind this revolution. We are a small, fast moving team of master generalists; that build, run, and operate the Magenta Marketing Platform. If you enjoy writing full stack code one day, working on a data pipeline the next and wrapping up your week writing infrastructure as code to deploy changes to the cloud we want to talk to you. We are looking for team members who can critically analyze a problem, collaborate with product management to find a solution, and own the delivery of that solution to the end customer.

• Develop data engineering solutions, Build and develop data pipelines to support Product, business and Data Science teams’ specific data needs.
• Assemble large, complex data sets that meet functional/ non-functional business requirements.
• Design and develop data architectures in on-premise, cloud and hybrid platforms.
• Data wrangling of heterogeneous data, exploration and discovery in pursuit of new business insights
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
• Actively contribute to the team’s knowledge and drive new capabilities forward.
• Mentor other team members in their efforts to build data engineering skillsets.
• Assist team management in defining projects, including helping estimate, plan and scope work.
• Prepare and contribute to presentations required by management
• Work with data and analytics experts to strive for greater functionality in our data systems.
• Also responsible for other Duties/Projects as assigned by business management as needed.
• Also responsible for other Duties/Projects as assigned by business management as needed.

Qualifications:

• Bachelor's Degree Computer Science, Statistics, Informatics, Information Systems or another quantitative field Required
• 5 years developing cloud solutions using data series; experience with cloud platforms (Amazon Web Services, Azure, or Google Cloud)
• Hands-on development using and migrating data to cloud platforms
• Experience in SQL, NoSQL, and/or relational database design and development
• Advanced knowledge and experience in building complex data pipelines with Python, Experience in languages such as Python, Java, Scala, and/or Go

Knowledge Skills and Abilities

• Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
• Ability to manage work across multiple projects, good organizational skills.
• Passionate about learning new technologies.
• Hands on Experience building production data pipelines, ideally using one or more distributed processing frameworks such as spark or Hive/Hadoop. Expertise in different types of Databases (Redshift, Snowflake) and query engine (SQL, Spark SQL, Hive)
• Strong Coding skills in Python, Java or Scala and SQL. Hands on experience building data engineering tooling in Cloud environment and some experience with building Airflow/ADF orchestrations.
• Creating automated code deployment/integration, Design CI/CD etc. is required.
• Analytical Approach to Problem-Solving; Ability to Use Technology To Tackle Business Problems
• Passionate About Learning New Technologies
• 4+ Years Developing Data Engineering Solutions. Experience working with Cloud Platforms (Amazon Web Services, Azure, Or Google Cloud) is a must.