Lead Big Data and Risk Software Engineer

Key Skills

Apache Hadoop, Apache Hive, Apache Kafka, Big Data Technology, Data management, Database, Debugging, Java Programming, Linux Operating system, Mentoring, MongoDB, NoSQL, Python Programming, RESTful, Risk Analytics, Software Development, Spark Core, XML-JSON

Job Description

The Risk Data Services is a horizontal function within Risk Technology organization and is responsible for delivering data consistently across Risk. The Risk Data Services team is seeking a Lead Software Engineer.

In this role, you will:

• Lead complex technology initiatives including those that are companywide with broad impact
• Act as a key participant in developing standards and companywide best practices for engineering complex and large scale technology solutions for technology engineering disciplines
• Design, code, test, debug, and document for projects and programs
• Review and analyze complex, large-scale technology solutions for tactical and strategic business objectives, enterprise technological environment, and technical challenges that require in-depth evaluation of multiple factors, including intangibles or unprecedented technical factors
• Make decisions in developing standard and companywide best practices for engineering and technology solutions requiring understanding of industry best practices and new technologies, influencing and leading technology team to meet deliverables and drive new initiatives
• Collaborate and consult with key technical experts, senior technology team, and external industry groups to resolve complex technical issues and achieve goals
• Lead projects, teams, or serve as a peer mentor

Required Qualifications, US:

• 5+ years of Software Engineering experience, or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, education
• 5+ years of experience on design, code, test, debug, and document for projects and programs
• 5+ years of experience in gathering requirements, defining business processes, identifying and analyzing risks using appropriate templates and analysis tools.
• 3+ years of experience integrating with RESTful API.
• 2+ years of hands-on experience in Hadoop – Spark, Scala, Java, JSON, Hive, Hbase, Python or Linux shell scripting.
• 2+ years of NoSQL DB experience such as Couchbase/MongoDB

Required Qualifications, International:

• Experience in Software Engineering, or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, education

Desired Qualifications:

• 5+ years of operational risk or credit risk or compliance domain experience.
• Prior experience as a Software Engineer at Wells Fargo
• Bachelor's degree in Computer Science or similar

Job Expectations:

• Design and implement Automated spark-based framework to facilitate data ingestion, transformation and consumption.
• Architect and design Hortonworks Big Data platform that leverage distributed computing and distributed storage.
• Implement multiple data engineering pipelines using Hadoop ecosystem tools such as Hive and Spark in order to process and publish data for Regulatory reporting.
• Implement security protocols such as Kerberos Authentication, Encryption of data at rest, data authorization mechanism such as role-based access control using Apache ranger.
• Design and develop automated testing framework to perform data validation.
• Design and build high performing and scalable data pipeline platform using Hadoop, Apache Spark, MongoDB, Kafka and object storage architecture.
• Work with Infrastructure Engineers and System Administrators as appropriate in designing the big-data infrastructure.
• Work with DBAs in Enterprise Database Management group to troubleshoot problems and optimize performance.
• Support ongoing data management efforts for Development, QA and Production environments.
• Utilizes a thorough understanding of available technology, tools, and existing designs.
• Leverage knowledge of industry trends to build best in class technology to provide competitive advantage.
• Acts as an expert technical resource to software engineers and senior software engineers in the development, testing, and implementation process.