Big Data Engineer

Key Skills

Java Programming, Python Programming, C++, C Programming, SQL, Apache Hadoop, Scala Programming, Machine learning techniques, Data science techniques, PyTorch, TensorFlow, MapReduce, R Programming

Job Description

Join the Data Technologies team as a Data Engineer

When was the last time you bragged about where you work? At CARFAX, we do it every day. Why? Because we’re proud to work for a company with a strong mission and trusted brand. We’re proud to work with people who care about what they do and work hard every day to deliver their best.

And just announced, the return of the 4-day work week from May 30th to Labour Day! We’ve created the type of company culture where the term, “work- hard play- hard” isn’t just a catchy saying, it’s part of the

As a Data Engineer on one of our core Data Technologies teams, you will work collaboratively with a team of software and data engineers to design, build and scale ‘Big Data’ pipelines to support the Carfax History Based Value product. Our team aggregates data across hundreds of billions of records every day. If you’re passionate about data and analytics and love the idea of working with others who share your passion, you’ll feel right at home as a data engineer on the Data Technologies team! 

As a Data Engineer you will:

• Work in a team environment using Agile practices
• Build high performing data pipelines to extract meaningful insights from billions of CARFAX vehicle records
• Utilize Test Driven, Paired Programming and Continuous Integration development methods
• Constantly learn from and educate others to improve yourself and CARFAX 

 As a Data Engineer you will be expected to have:

• Ability to solve complex problems and learn from mistakes
• Experience working in a collaborative team environment 
• Strong analytical skills related to working with both structured and unstructured datasets
• Experience writing complex queries in relational and non-relational databases

Preferred Skills, Knowledge, and Experience

• Expertise in Scala. Java and shell scripting experience is also preferred 
• Experience with Hadoop data tools: MapReduce, Hive, Apache Spark, Apache Sqoop 
• Knowledge of data pipeline management applications: Apache Airflow, BMC Control-M 
• CI/CD pipeline automation experience with Jenkins, GitLab, or AWS Code Build, 
• DevOps expertise with tools such as Terraform, Artifactory, Hashicorp Vault, AWS CloudFormation
• SQL & NoSQL experience with MongoDB, Oracle, MySQL or other databases
• Familiarity with AWS data engineering technologies: Glue, EMR, AWS Lake formation, S3 as Data Lake, RedShift