Sr Data Engineer ML

Key Skills

Java Programming, Design of Experiment, Python Programming, Database, Framework, SQL, DevOps, Azure, AWS, PowerBI, SQL Server, Big Data Technology, Data Modelling, Snowflake, Docker, Kubernetes-K8s, Continuous Integration & Continuous Delivery - CI/CD, Terraform, Postgres, PySpark, Linux Operating system, SPARK Programming, Amazon Simple Storage Service (S3), Data pipelines, ETL frameworks

Job Description

Our company are leaders delivering fuel pricing data, market intelligence and analytics to the global Energy industry. We believe in having the best technology capabilities to deliver our products to our customers.

OPIS, a part of Dow Jones, is expanding its Data Science and Machine Learning (MLOps) Engineering team. You will play a critical role in the design and development of a world-class Data Science and MLOps platform. We encourage creativity, autonomy, and flexibility to allow you to bring your ideas to life!

To fit this role, we are looking for a mid-level data engineer with a unique mix of technical capabilities, strong data analysis and collaboration skills.

Your Main Responsibilities

• Design and develop complex code in SQL or Python to optimize data pipelines for ML analysis and modeling
• Write PySpark code to transform our model algorithms to production-ready code
• Work with the latest AWS cloud computing and data warehousing technologies such as Snowflake
• Work with the latest AWS services to deploy and run our ML code via CI/CD pipelines and containers
• Be part of a diverse, global Agile team, employing best practices for the DSML Lifecycle Framework
• Be proud of both your own growth and your team’s success!

Who We Are

Our company are leaders delivering fuel pricing data, market intelligence and analytics to the global Energy industry. We believe in having the best technology capabilities to deliver our products to our customers.

Your team focuses on developing the data infrastructure and platform to support our large volume data analytics processing, AI, data science, and machine learning projects.

Your Expertise

You have:

• 7+ years of working experience in data management technologies (Microsoft SQL Server, Postgres, advanced SQL coding, relational database design, data warehousing)
• 5+ years with writing SQL ETL processes
• 3+ years coding in Python or PySpark
• 3+ years developing Big Data and ML technologies (Spark, AWS SageMaker) to optimize ML model processing (training, evaluation, and inference)
• 3+ years in cloud computing especially AWS
• Experience working in a Linux and Windows environment

Bonus if you also have experience in:

• Writing infrastructure as code for AWS (Terraform, Step Functions, CloudWatch)
• Working with SnowFlake cloud data warehouse, parquet file formats, AWS S3 storage
• Creating CI/CD deployment pipelines in Microsoft Azure DevOps
• Working with containers (Docker, Kubernetes)
• Deployments in MLOps software (AWS SageMaker, Databricks)
• Working with BI software (Microsoft Power BI)
• Educational background in computer science, engineering, or data analytics

What We’re Offering

• Great salary
• Career growth opportunities and training programs
• Flexibility in work from home
• A diverse and flexible work environment
• Team building and social events
• Generous vacation allotments

Dow Jones , Making Careers Newsworthy

All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, protected veteran status, or disability status. EEO/AA/M/F/Disabled/Vets .

Dow Jones is committed to providing reasonable accommodation for qualified individuals with disabilities, in our job application and/or interview process. If you need assistance or accommodation in completing your application, due to a disability, please reach out to us at TalentResourceTeam@dowjones.com . Please put “Reasonable Accommodation in the subject line.

Business Area: OPIS

Job Category: IT Development Group

Union Status:

Non-Union / No clear and likely Internal Candidate (Employee Only) has been identified

Req ID: 31201