Data Engineer

Key Skills

Java Programming, Python Programming, C++, C Programming, SQL, Cloud computing, Scala Programming, Machine learning techniques, Data science techniques, MATLAB Programming, PyTorch, TensorFlow, R Programming

Job Description

Data Engineer

M&S is ready to push boundaries to lead the industry into a greener, speedier, more inspiring digital era. That’s why we’re revolutionising how we work and offering our most exciting opportunities yet! There’s never been a better time to be part of our team.

This is a great opportunity to join the Digital and Data team, to support the delivery of our transformation goals, be a role model for digital behaviours and championing a culture of data led decision making.

We are looking for an experienced engineer to support the delivery of our data transformation. You will develop data solutions to deliver capabilities and support the migration onto our new cloud-based data platform.

The key objective of the role is to develop high quality, automated pipelines which will ingest, transform, and create structured and performant data sets on our new cloud platform.

It is important to develop strong working relationships with technical leads, data modellers and other engineers to collaborate on cross-data team priorities and support the continuous maturity and development of our engineering community.

Key accountabilities & responsibilities

• Design, develop, and maintain scalable data pipelines that adhere to ETL principles and business goals.
• Understand and identify data requirements and incorporate business logic to develop transformations.
• Develop relational data models designed for usability and optimise for performance.
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Azure data technologies.
• Advise and collaborate with data engineers, data analysts and data quality analysts on data standards and practices.
• Participate in a culture of sharing, re-use, design for scale, and operational efficiency of data.
• Partner with business domains and architects to develop solutions for strategic projects and initiatives.
• Analyse current business practices, processes and procedures and identify future opportunities for maximising Microsoft Azure data & analytics services.
• Work with the data architect to build the core data model for the organisation both from an operational and analytical perspective.
• Work with data and analytics experts to strive for greater functionality in our data systems.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Support the build of analytical tools that utilise the data pipeline to provide practical insights, enabling data driven decision making, operational efficiency and other key business performance metrics.
• Identify and problem solve data accuracy and availability challenges.
• Develop in line with standard methodology around CICD, code frameworks and testing.
• Ensure build of data sets follow GDPR and CISO policies.
• Build relationships with key business partners and users.
• Collaborate and support the wider engineering team to ensure standards are maintained and knowledge is shared.

Key Experience

• Minimum 2+ years of experience with ETL tools, Databricks, SQL, Proven experience of building end-to-end ETL pipelines
• Expertise using Big Data technologies; and preferred experience in cloud-based data technologies and data warehousing design principles, preferably Azure
• Experience with programming languages, Python preferred.
• Hands on experience with schema design and dimensional data modelling, automation processes and version control tools
• Experience with DevOps supporting production pipelines
• Mentoring junior members of the team
• Hands on experience with modern data storage systems (e.g. ADLS)
• Customer centric, passionate about delivering phenomenal digital products and services
• Passionate about continuous improvement, collaboration, and great teams
• Strong problem-solving skills coupled with good communication skills
• Open minded, inquisitive, life-long learner

Desirable

• Degree in computer science or engineering
• Experience in retail
• Experience in the following technology: Azure, Databricks, ADF, DevOps, Git Hub, Synapse, Starburst, Alation, Ataccama
• Experience in languages
• PySpark or Python
• SparkSQL or SQL

About M&S

Marks & Spencer strives to be an inclusive organisation, trusted and loved by our colleagues, customers and suppliers. Join us and make an immediate impact. We are committed to an active Inclusion, Diversity and Equal Opportunities Policy, which starts with our recruitment and selection process, and we are happy to talk smart/flexible working.