Senior Data Engineer

Key Skills

Aartificial intelligence, Amazon Athena, Amazon Simple Storage Service (S3), Apache Airflow, AWS Glue, AWS Sagemaker, Continuous Integration & Continuous Delivery - CI/CD, Data Governance, Data security, Data Warehousing, Docker, Effective communication skills, ETL frameworks, Machine learning techniques, Python Programming, SQL

Job Description

You will be part of our Finance Advanced Analytics (FAA) team in one of our locations in India. FAA develops innovative data science models and tools for a wide variety of internal stakeholders, helping to solve some of the firm’s most difficult global and regional data-driven operational challenges.

As a data engineer in the FAA group, you will support impact-oriented internal firm use cases by partnering in all data-related aspects of the project lifecycle, from data exploration to pipeline deployment. While developing bespoke data solutions, you will collaborate with colleagues who have specific expertise in each relevant area and can act as a translator between the data science/engineering side and the business side.

What You'll Do

You will be primarily responsible for data sourcing and integration, pipeline and process automation, and critical cloud infrastructure development. You will work closely with team members and our stakeholders to create valuable technology-driven products and capabilities across the finance space. This will include working with data owners and knowledge experts to procure and understand relevant data sets. You will be responsible for delivering clear, well-structured communication to internal customers in partnership with FAA teammates.

You will have the opportunity to leverage your talent for using technology to achieve real impact, and you will proactively stay up to date on technology trends with the support and encouragement of team and firm leadership.

This role will provide you with the chance to further strengthen and develop your skills through a blend of data structuring, analyses, and problem solving, with a strong focus on building strong and robust foundations for our firm-changing products.

Your primary responsibilities will include:

• Analyzing use case needs to design appropriate data storage and provision solutions from firm data systems to team data infrastructure
• Building appropriately scaled, highly available, fault-tolerant data pipelines and ETL processes to drive data flow into and within AWS services
• Identifying and maintaining best practices regarding data management, security, cost control mechanisms, source control, code deployment, and load balancing
• Migrating existing on-premises or “other-cloud” workloads, products, and services to improved AWS-based locations and services
• Actively driving Continuous Integration and Continuous Delivery (CI/CD) based application development, including managing source control systems, virtual servers, and scripting
• Establishing and maintaining cloud instances for “sandbox,” development, and production environments in partnership with our Technology & Digital Data Lake team
• Configuring and maintaining containerized (e.g. Docker) instances, if needed
• Developing SQL scripts and procedures in Snowflake
• Coordinating information sharing with core SAP transformation team members on behalf of the GFA and FAA teams
• Adapting existing T&D Data Lake pipelines and tables in response to the SAP transformation, preserving Finance and non-Finance data availability to both FAA and GFA teams

Qualifications

• Implementation and architectural experience with big data, data warehousing, business intelligence, or AI/ML engineering
• Experience with building and automating ETL data pipelines
• Experience with operating and managing AWS ecosystem, including users/access, configuration, and tools including Athena, S3, Elastic Beanstalk, and Glue
• Familiarity with SageMaker is a plus
• Experience with constructing relational and dimensional data models and identifying linkages across disparate real-world data sets
• Knowledge of data security and governance; professionalism in dealing with confidential information
• Fluency in Python and SQL
• Experience with open-source tools such as Airflow
• Strong project management, teamwork, and problem-solving skills, including optimizing new processes
• Excellent written and verbal communication skills; ability to describe analytic processes and insights to both technical and management stakeholders
• Distinct customer focus and quality-first mindset
• Integrity, maturity, and adherence to the highest professional standards