Data Engineering Lead - NextGen - VP

Key Skills

Java Programming, Python Programming, MongoDB, SQL, XML-JSON, AWS, Big Data Technology, Apache Hadoop, NoSQL, Apache Hive, Apache Impala, Amazon EMR, SPARK Programming, Storm, HBase, Amazon Simple Storage Service (S3), Hortonworks

Job Description

We are looking for a highly motivated, self-starter mid-senior level Data Engineering Lead with a passion for data and with the ability to drive multiple projects on our Big Data Platform. This role requires 8-10 years’ experience with hands-on experience on Big Data & NoSQL platforms. We are looking for somebody who can not only manage a small team if needed but also be technically involved in the day-to-day work and contribute technically as and when needed. Experience working in the financial industry within the Institutional business would be a plus. This is a hands on development role that will offer exposure to the full development cycle, whilst working closely with our business and technology stakeholders.

As a key member of the data engineering team, you will work closely with full stack data engineers, product managers, cross-funcability to drive multiple projects on our Big Data Platform. This role requires 8-10 years’ experience with hands-on experience on Big Data & NoSQL platforms. We are looking for somebody who can not only manage a small team if needed but also be technically involved in the day-to-day work and contribute technically as and when needed. Experience working in the financial industry within the Institutional business would be a plus. This is a hands on development role that will offer exposure to the full development cycle, whilst working closely with our business and technology stakeholders.

As a ketional partners and the larger technology organization to get high-quality products and new features built, tested, and released on time for Data Technology, part of Treasury & Trade Services at Citi.

• You will be responsible for many fast-changing, moving parts and get them to come together as a product.
• You will need excellent communication and collaboration skills.
• You will be responsible for identifying and managing risks, making sound judgments about quality, and speed of deliverables and deployment to production.
• You will be a key player in our Data transformation to digitize our business
• You will be responsible for leading development on Vanguard the TTS Big Data platform

Background

The Vanguard Data Management Team supports data ingestion, data quality and data conformance on the Big Data platform (Cloudera). The team also manages several NoSQL data platforms and operational real-time event based processing and compute applications as well as analytics consumption use cases including machine/deep learning.

The technology team is responsible for building and maintaining a multitude of applications including:

• Batch ingestion using Talend and other Big data tools
• Real-time ingestion/stream processing and data distribution via Big Data APIs
• Building data caching layer on NoSQL data platforms for fast response times
• Build out canonical models and data conformance
• Implement best in class data management and data ingestion.
• Build and implement best in class Data Quality tools for monitoring of Critical Data Elements

Core Responsibilities

This role is for a Data Engineering Lead to work on the Vanguard Big Data Platform. The team is responsible for the maintenance and development of leading Big Data initiatives and use cases providing business value.

• Interface with product teams to understand their requirements to build the ingestion pipelines and conformance layer for consumption by business.
• Work closely with the data ingestion team to track the requirements and drive the build out of the canonical models
• Provide guidance to the data conformance team for implementing the requirements/changes/enhancements to the conformance model
• Do hands-on development as part of the conformance team to deliver the business requirements
• Manage the workload of the team and the scrum process to align it with the objectives and priorities of the product owners
• Participate in data management activities related to the Risk and Regulatory requirements as needed.

Core Skills

The Data Engineering lead will be working very closely with and managing the work of a team of data engineers working on our Big Data Platform. The lead will need the below core skills –

• Strong solid understanding of the Big Data architecture and the ability to trouble shoot performance and/or development issues on Hadoop (Cloudera preferably)
• Hands-on experience working with Hive, Impala and HBase for data curation/conformance related work
• Strong proficiency in Spark for development work related to curation/conformance. Strong Scala developer (with previous Java background) preferred.
• Experience with Spark/Storm/Kafka or equivalent streaming/batch processing and event based messaging
• Strong data analysis skills and the ability to slice and dice the data as needed for business reporting
• Experience working in an agile environment with a fast paced changing requirements.
• Excellent planning and organizational skills
• Strong Communication skills

Additional Requirements (good to have)

• Experience working with NoSQL data platforms like Cassandra, MongoDB etc.
• ·Cloudera/Hortonworks/AWS EMR, S3 experience a plus
• ·Experience with the AWS or GCP tech stack components
• ·Relational and NoSQL database integration and data distribution principles experience
• ·Experience with API development and use of JSON/XML/Hypermedia data formats
• ·Analysis and development across Lines of business including Payments, Digital Channels, Liquidities, Trade
• ·Cross train and fertilize functional and technical knowledge,
• ·Align to Engineering Excellence Development principles and standards.
• ·Promote and increase our Development Productivity scores for coding
• ·Fully adhere to and evangelize a full Continuous Integration and Continuous Deploy pipeline