Data Engineer

Key Skills

Java Programming, Python Programming, C++, C Programming, SQL, Apache Hadoop, Scala Programming, Machine learning techniques, Data science techniques, PyTorch, TensorFlow, MapReduce, R Programming

Job Description

We’re looking for a dynamic data engineer with Apache Spark and AWS experience to join the data science team at Gainwell Technologies. You will have the opportunity to work in a cross-functional team to define, design and deploy frameworks for data extraction, cleansing, transformation, and normalization using the Apache Spark framework to support the analytic missions of Gainwell and its clients.

Job Description

Essential Job Functions

• Design, develop and deploy data pipelines including ETL-processes for extracting, transforming, normalizing, and delivering data using Apache Spark Framework.
• Build automated data pipelines to support AI/ML models.
• Monitor, manage, validate, and test data extraction, movement, transformation, loading, normalization, cleansing and updating processes. 
• Coordinates with data science, data engineering, infrastructure, end users, and other teams to understand data needs and delivery of data with a focus on data quality, data reuse, consistency, security, and regulatory compliance.
• Collaborate with team-members on data models and schemas in our data warehouse.
• Conceptualize and visualize data frameworks.
• Communicate effectively with various internal and external stakeholders.

Basic Qualifications

• Bachelor's degree or equivalent combination of education and experience
• Bachelor's degree in computer sciences or related field preferred
• Nine or more years of relevant experience or equivalent education in ETL Processing/data architecture or equivalent.
• 2+ years of experience working with big data technologies such as Hive/Spark.
• 1+ years of experience in the Apache Spark/DataBricks framework (Python/Scala)
• Experience working with different database structures (e.g., transaction based vs. data warehouse)

Other Qualifications

• Strong project planning and estimating skills related to area of expertise
• Strong communication skills
• Good leadership skills to guide and mentor the work of less experienced team members
• Ability to be a high-impact player on multiple simultaneous engagements
• Ability to think strategically, balancing long and short-term priorities

This position is available for remote, home-based work within the United States.