Staff Data Engineer

Key Skills

Apache Hadoop, C Programming, C++, Data science techniques, Java Programming, Machine learning techniques, MapReduce, Python Programming, PyTorch, R Programming, Scala Programming, SQL, TensorFlow

Job Description

At bp, we’re reimagining energy for people and our planet. With operations working across almost every part of the energy system, we’re leading the way in reducing carbon emissions and developing more sustainable methods for solving the energy challenge.
In innovation & engineering we work with innovative technologies and big-data to help tackle society’s big energy challenges. Choose to join bp and you’ll be on the front line of the energy transition, collaborating with world-class technologists, engineers, and disrupters to reimagine energy and carve out a career packed with purpose, passion, and learning.

Role Synopsis

dataWorx is the name for the data team that is responsible for all data within these areas:

• Production & Projects including Health, Safety, Environment & Carbon
• Refining & Operations
• Wells & Subsurface
• Business Services including Finance, Procurement, People & Culture, Performance Management
• Strategy & Sustainability

We are developing deep data capability to transform the access, supply, control and quality to our vast and ever-growing reserves of data. The dataWorx team covers many data sub-disciplines, including data science, analytics, engineering and management as well as specialist areas such as geospatial, remote sensing, knowledge management and digital twin.
We are looking for outstanding data engineers to power this transformation and unlock the value of our digital assets to power our journey to net zero emissions and build a new, sustainable bp.

Key Accountabilities

• Leads, grows and develops a team of data engineers that writes, deploys and maintains software to build, integrate, manage, maintain, and quality-assure data at bp
• Lead, design, develop, and deliver large-scale data ingestion, data processing, and data transformation projects on Azure
• Mentor and share knowledge with customers as well as provide architecture reviews, discussions, and prototypes
• Work with customers to deploy, manage, and audit best- practices for cloud products
• Engages and drives an inclusive environment with team and partners through the quality of interactions and collaboration across multiple business entities
• Architects and designs reliable and scalable data infrastructure
• Advocates for and ensures their team adheres to software engineering best- practices (e.g. technical design and review, unit testing, monitoring, alerting, checking in code, code review)
• Responsible for deploying secure and well-tested software that meets privacy and compliance requirements
• Responsible for service reliability and following site-reliability engineering best- practices: on-call rotations for services they oversee, responsible for defining and maintaining SLAs
• Actively contributes to improve developer velocity.

Desirable Education

• BS degree in computer science or related field

Desirable Experience and Capability

• 5+ years of deep and hands-on experience designing, planning, product-ionizing, maintaining and documenting reliable and scalable data infrastructure and data products in complex environments
• 2+ years of experience leading, growing and developing a data engineering team of around 7-30 people

Required Criteria

• Deep and hands on experience designing, planning, product-ionizing, maintaining and documenting reliable and scalable data infrastructure and data products in complex environments
• Hands on experience with:
• Databricks and using Spark for data processing
• Configuring Delta Lake on Azure Databricks
• Python, Scala, SQL
• C#, ASP.NET, MVC, .NET Core, .Net Framework (4.6) JSON, and API development (optional)
• Azure Data Factory
• Azure Data Lake, Azure SQL DB, Synapse, and Cosmos DB
• Data Management Gateway, Azure Storage Options, Stream Analytics and Event Hubs
• Designing data solutions in Azure incl. data distributions and partitions, scalability, disaster recovery and high availability
• data governance, security, and privacy
• Understand a broad spectrum of technology in order to provide detailed technical design which meets customer requirements
• Experience architecting solutions in the cloud
• Advanced hand-on experience with different query languages
• Experience designing and implementing large-scale distributed systems
• Stakeholder management and ability to lead large organizations through influence

Why join us

At bp, we support our people to learn and grow in a diverse and challenging environment. We believe that our team is strengthened by diversity. We are committed to fostering an inclusive environment in which everyone is respected and treated fairly.
There are many aspects of our employees’ lives that are important, so we offer benefits to enable your work to fit with your life. These benefits can include flexible working options, a generous paid parental leave policy, and excellent retirement benefits, among others!
We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment. Please contact us to request accommodation.