Data Engineer with Analytics Background

Key Skills

Algorithms, Amazon EMR, Analytical and Problem solving, AWS, Data structuring, Databricks, Java Programming, Optimization, Python Programming, Scala Programming, Scrum Agile Methodology, Spark Core

Job Description

The Data Engineer will be responsible for independently analyzing, developing, testing, and supporting complex data products and applications core to our business.

Primary Responsibilities

• Provide operational and production support for our syndicated data products.
• Design and develop scalable data processing pipelines and applications using big data technologies (Databricks/Spark, Hive, AWS data and analytics services, etc.)
• Daily activities will involve development, testing, meeting with technology leadership to work through technical designs, supporting other developers on the team with their deliverables, and participating in Agile (SCRUM) ceremonies
• Participate in problem identification and requirement gathering by working closely with product owners or analysts to understand business and functional requirements
• Work with Data Science group to profile large data sets, and define and implement analytical models
• Convert existing code / processes in Oracle into Databricks (PLSQL, Python, Scala

Skills, Education and Experience

Required:

• Bachelor’s degree in Computer Science, Engineering, Statistics, Applied Mathematics or equivalent work experience in the data engineering field
• Experience working with large and diverse data sets, and building and maintaining data processing systems
• Proficiency in at least one of the following languages: Python, Scala , PLSQL or Java
• Ability to apply analytical and problem-solving skills to solve complex design and coding problems
• Ability to learn and adapt to continually evolving technologies in the big data ecosystem
• Ability to manage multiple projects and meet deadlines 

Highly preferred:

• Hands-on experience with the Databricks platform or Spark on AWS EMR
• In-depth knowledge of Algorithms, Data Structures, and Performance Optimization Techniques
• Experience with AWS data services
• Background in Scrum/Agile development methodologies