Senior Data Lake Engineer - Data Lake Engineering - Jersey City

Key Skills

AWS, Apache Flink, Java Programming, MapReduce, Python Programming, SPARK Programming, SQL

Job Description

Data Lake Engineers are innovators and problem-solvers, building solutions in risk management, big data, mobile and more. We look for creative collaborators who evolve, adapt to change and thrive in a fast-paced global environment. Our team, Data Architecture, is focused on the firm’s data strategy.

Data Lake Engineering is responsible for the Firm’s strategic repository for enterprise data. Technology teams across the Firm are clients, participating in providing and consuming data to and from the lake. Developers on the team create and manage the software that manages the data in the lake, ensure entitlements are enforced appropriately, data is milestoned, and is available for query on multiple target warehouse platforms.

YOUR IMPACT

As a Senior Data Lake Engineer, make data a strategic asset for the enterprise by providing a platform that enables the structuring, management, integration, control, discovery, usage, and governance of our Data Assets.

HOW YOU WILL FULFILL YOUR POTENTIAL

• All roles in Data Architecture/Data Lake Engineering offer the opportunity to work directly with various senior stakeholder across Technology Division and their Business sponsors
• The Data Lake is being adopted by technology teams across the Firm at a very high rate. As a result, the platform is still growing and evolving. As a developer on the Data Lake team, you will work with the latest technologies such as Apache Spark, Flink, Hive, Presto, Kafka, Elastic Search, and Akka to build complex distributed applications to handle large data sets
• Collect requirements, and assess technical feasibility of their requests of your clients and adjacent technology platforms
• Participate in design, build out, and maintenance of a distributed, multi-region, scalable, resilient platform. Collaborate with core engineering consumers and the BU aligned development community at large to analyze and resolve escalations
• Collaborate with other team members across region
• Your clients’ background range from very technical engineers, to more functional and business facing analysts and managers. Partner with other engineering teams to understand technology offerings and how they can be applied and leveraged as part of our solutions

BASIC QUALIFICATIONS

• 6 to 8+ years of experience with software development is needed
• 3+ years of experience with Java with a good grasp of Java development
• Object Oriented Analysis and Design skills and familiar with design patterns
• Good understanding of distributed system
• Good understanding of database technologies
• Working knowledge of scripting languages, Linux, Networking protocols, security and file systems
• Strong technical skills, analytical mindset, self-motivated, independent, creative, can solve interesting and sometimes difficult technical problems under time pressure and resource constraints
• Commercially focused; seeks to understand the requirements and how they will benefit our clients, stakeholders, and business
• Experience with all stages in software development lifecycle: inception, analysis, design, development, review, testing, and deployment
• Experience building and sustaining long-term relationships with clients and colleagues in a diverse global organization
• Judgment to prioritize and escalate issues in order to influence objectives and outcomes
• Excellent written and verbal communication skills, including experience working directly with both technical and non-technical stakeholders

PREFERRED QUALIFICATIONS

• Bachelor’s Degree and/or Master’s Degree in Computer Science or related field is needed
• Experience with Hadoop and its ecosystem, including Spark, MapReduce, and Flink
• Experience with AWS
• Experience with developing Elasticsearch, and Sybase IQ