Senior Data Engineer

Key Skills

.NET, Agile methodologies, Apache Kafka, API, Azure, Data Architecture, Data pipelines, Data science techniques, Databricks, Design, DevOps, ETL/ELT technologies, Machine learning techniques, NoSQL, Product Management, PySpark, Python Programming, SQL, UI design, UX design

Job Description

About us

Come be a part of driving National Grid’s digital transformation! We are digital creators, continuous learners and daring innovators. We leverage digital innovative ways to create products and catalyze the transformation of National Grid's business units into more agile and digitally native organizations in our shared purpose of bringing energy to life. 

At National Grid, we keep the lights on and homes warm. But it’s so much more than that. We keep people connected and society moving. This is no easy feat, and it takes all of us. But National Grid supplies us with the environment to make it happen. As we generate momentum in the energy transition for all, we don’t plan on leaving any of our customers in the dark. So, join us and help bring energy to life. 

About the role

As a Senior Data Engineer, you will be involved in the product development process from ideation to launch, working closely with software engineers, data scientists, product managers, UI/UX designers, and key stakeholders across the organization to deliver the best digital experiences for our users. You will actively contribute to the success of the team by innovating, participating in technical discussions, and contributing to the development of products that will transform the organization.
You will design and develop highly scalable and extensible data pipelines, which enable collection, storage, distribution, modeling, and analysis of large data sets to deliver data that is accessible and of the highest quality.  This position requires an innovative software engineer who is passionate about data & data quality and will contribute to the modernization of our architecture and tools to help increase our output, scalability, and speed.

What you'll do

• Implement data orchestration pipelines, data sourcing, cleansing, and QC processes 
• Support data architects when designing data architectures 
• Contribute to software verification plans and quality assurance procedures
• Design and develop data integrations and a data quality framework. Write unit/integration/functional tests and document work.  
• Collaborate with business and technical stakeholders on the development of digital products
• Consult with end users to refine, redesign, and perfect large-scale solutions
• Contribute to IT standards, procedures, and process 
• Actively participate in the design of software solutions 
• Document and maintain software functionality

What you'll need

• Experience with large scale, full lifecycle data implementation projects 
• Experience in Python and PySpark. 
• Experience in deployment of data pipelines in Azure (Databricks, Event Hubs, Kafka)
• Experience developing integrations across multiple systems and APIs.
• Experience with or knowledge of Agile software development methodologies 
• Experience with cloud-based databases, specifically Azure technologies (e.g., Azure Data Factory, Azure DevOps and Azure Functions)
• Experience using SQL queries as well as writing and perfecting SQL queries in a business environment with large-scale, complex datasets.
• Developed ETL/ELT workflows with a modern programming language (Python, .NET) 
• Experience designing data schemas and operating SQL/NoSQL database systems is a plus
• Experience in Data Science, Machine Learning is nice to have
• Demonstrate flexibility to respond to the changing needs of the business

What you'll get

• 28 days holiday, plus eight statutory days – and options to buy extra holiday days
• Pension and share option scheme
• Close mentorship and regular feedback
• You will also have access to a number of other flexible benefits such as share incentive plan, company car, and technology schemes, support via employee assistance lines and matched charity giving name to a few.