Lead Data Engineer

Key Skills

Java Programming, Oracle, Python Programming, SQL, Data science techniques, Teradata

Job Description

Job Summary:

The primary purpose of this role is to provide consultation and technical advice on translating business requirements and functional specifications into logical program designs. This includes facilitating the implementation and maintenance of complex business and enterprise data solutions to ensure successful deployment of released applications. This role serves as a technical expert for project teams throughout the implementation and maintenance of business and enterprise Data or Platform solutions. In addition, this role personally develops and delivers code modules, stable application systems, and integrated enterprise Data or Platform solutions within various computing environments.

Key Responsibilities:

• Resolves complex problems spanning multiple applications to drive improvements across portfolios; leads projects with significant business implications, participates in planning and priority estimates, and advises Software Engineers; executes with limited guidance
• Leads development of integrated business and/or enterprise application solutions to ensure specifications are flexible, scalable, and maintainable and meet architectural standards
• Educates others on current architectural standards and guidelines to drive efficiency when leading the design efforts of complex business and enterprise software solutions
• Supports systems integration testing (SIT) and user acceptance testing (UAT) for large, complex, cross-functional application initiatives by providing insight to testing teams in order to ensure the appropriate depth of test coverage
• Mentors and advises others in all software development lifecycle phases by applying and sharing an in-depth understanding of company and industry methodologies, policies, standards, and controls
• Communicates changes in software architecture and coaches others to apply this understanding to software solutions; resolves escalated issues
• Leads efforts to improve engineering, test, and operational excellence best practices.
• Solves complex cross-functional architecture/design and business problems; solutions are extensible; works to simplify, optimize, remove bottlenecks, etc.
• Mentors and advises others, sharing an in-depth understanding of company and industry methodologies, policies, standards, and controls

Data Engineering Responsibilities

• Designs and leads the development, maintenance, and enhancements of data lake developments / data ingestion solutions across various business domains by working closely with a matrixed data engineering team to deliver data pipelines on on-prem and cloud infrastructure; establishes frameworks and reusable components for repeatable data ingestion processes; proactively identifies opportunities to build data lake ahead of business needs
• Excels in one more domain; understands pipelines and business metrics and champions one or more cloud-based data development frameworks and associated tools
• Builds, tests and enhances data curation pipelines integration data from wide variety of sources like DBMS, File systems, APIs and streaming systems for various KPIs and metrics development with high data quality and integrity
• Supports the development of feature / inputs for the data models in an Agile manner; Hosts Model Via Rest APIs; ensures non-functional requirements such as logging, authentication, error capturing, and concurrency management are accounted for when model hosting
• Works with Data Science team to understand mathematical models and algorithms; educates and influences Data Scientists on how Data Engineering can help scale and deploy models; recommends improvements to analytic methods, techniques, standards, policies and procedures; participates in continuous improvement activities including training opportunities; trains others; uses expert technical competence to contribute to policy and procedure updates
• Ensures the definition, design, manipulation and administration of data and systems are secure and in accordance with enterprise governance by staying compliant with industry best practices, enterprise standards, corporate policy and department procedure; ensures compliance with governance and security standards; implements reusable design patterns and processes
• Maintains the health and monitoring of data engineering capabilities that span analytic functions by accumulating and grouping maintenance issues; ensures high availability of the platform, monitors workload demands; works with Infrastructure Engineering teams to maintain the data platform; works with management to align maintenance issues with program and project work; triage, accept/reject and assign maintenance issues; serves as an SME of one or more application
• Coordinates with Management and vendors on data engineering data and product roadmaps that align to business roadmaps

BI Engineering Responsibilities

• Designs and leads the development, maintenance, and enhancements of BI solutions across various business domains; provides direction to matrixed BI engineering resources to deliver reports on-prem and cloud infrastructure; creates frameworks and reusable BI components
• Excels in one or more domains and proactively identifies opportunities to build self-service and complex BI solutions ahead of business needs; develops expertise on cloud-based BI stacks
• Demonstrates database skill (Teradata/Oracl