Data Layer Engineering Lead

Key Skills

Agile methodologies, API, Azure, Big Data Technology, Continuous Integration & Continuous Delivery - CI/CD, Data Architecture, Data Engineering, Data Governance, Data Lake, Data Visualization, Data Warehousing, Databricks, Kubernetes-K8s, Lakehouse, Management

Job Description

• Data Layer Engineering Lead will own the delivery of a large, globally distributed and technically diverse team of engineers, working to build enterprise grade data platform. Service lines will build data solutions and consume data services from the Fabric Data Platform that caters to end-to-end data lifecycle. Data Layer Engineering Lead will be working in an complex environment and must align with a variety of stakeholders. The Engineering Lead will be responsible for mentoring the team and providing engineering thought leadership.

  Job Responsibilities:

  • Team Leadership: Lead and manage a diverse team of engineers (Data, Application, DevOps, and QA Engineers), providing guidance, mentoring, and support to ensure high performance and continuous improvement.
  • Engineering Delivery: Drive the successful delivery of data platform components and services, ensuring adherence to timelines, quality standards, and stakeholder expectations.
  • Technical Expertise: Provide technical leadership and expertise to develop data platform in data engineering, application engineering, DevOps and QA Engineering.
  • Stakeholder Management: Collaborate with a variety of stakeholders, such as product managers and architects to understand requirements, align priorities, and ensure smooth execution.
  • Team Development: Identify individual and team training needs, implement learning initiatives, and foster a culture of knowledge sharing and professional growth.
  • Process Improvement: Continuously improve development standards, processes, methodologies, and tools to enhance team efficiency and effectiveness.
  • Risk Management: Identify and mitigate risks associated with data platform development, ensuring compliance with security, data privacy, and regulatory standards.
  • Innovation and Adoption: Stay up to date with emerging technologies, industry trends, and best practices in engineering, and apply them to drive innovation.

  Technical/Functional Skills Required:

  • Expert in leading a product engineering team to develop an enterprise scale data platform
  • Expert in Big Data technologies such as Data Lake, Data Warehouse, Lakehouse, Data Mesh Architecture, Data Fabric etc.
  • Experienced in various data lifecycle stages like Extraction, Ingestion, Data Quality, Transformation, Storage, Consumption Mechanisms, Governance etc.
  • Expert in Microsoft Big Data stack, Synapse, Azure Databricks, Azure Data Lake Store, Azure EventHub, Azure Stream Analytics, Kubernetes, API management, APIs and Frontend technologies
  • Experience in data governance tools such as Immuta, Collibra, Microsoft Purview etc.
  • Experience of Master Data management (MDM), Metadata Management, Data Visualization and Data Governance (DG).
  • Broad understanding of various emerging data and analytics related tools and technologies.
  • Experience in enterprise grade Platform development
  • Experience in working with SAFe Agile methodology
  • Understanding of CI/CD automation and self-service provisioning
  • Understanding of Microservice based architecture
  • Understanding of APIs and Frond-end web development technologies

  Additional skill requirements:

  • Demonstrated experience working with a large globally distributed team, across cultures and time zones
  • Demonstrated ability to quickly grasp new technologies and embed in engineering execution
  • Should be able to adapt to constantly evolving requirements and priorities
  • Creative problem solving skills

  Job Requirements:

  Experience:

  20+ years demonstrable engineering experience

  10+ years of technology product and/or platform development experience

  10+ years of data domain industry experience

  10+ year of experience in managing a team of architects and engineers with a global presence.

  10+ years of experience in an SAFe Agile base delivery methodology

  5+ years of experience in performing analysis and design for Data Management and Data Driven projects.

  5+ years of experience in driving complex Data Pipeline solutions and architectures.

  5+ years of experience in driving Metadata architecture and Master Data Management Solutions.

  Education / Certifications:

  Computer science graduate or equivalent degree.

  Microsoft Azure certifications for fundamentals, Data engineering and Developer areas

  Databricks certification preferred