Engineering Director - Big Data

Key Skills

Apache Hadoop, C Programming, C++, Data science techniques, Java Programming, Machine learning techniques, MapReduce, Python Programming, PyTorch, R Programming, Scala Programming, SQL, TensorFlow

Job Description

At American Express, we know that with the right backing, people and businesses have the power to progress in incredible ways. Whether we’re supporting our customers’ financial confidence to move ahead, taking commerce to new heights, or encouraging people to explore the world, our colleagues are constantly redefining what’s possible — and we’re proud to back each other every step of the way. When you join #TeamAmex, you become part of a diverse community of over 60,000 colleagues, all with a common goal to deliver an exceptional customer experience every day. We back our colleagues with the support they need to thrive, professionally and personally. That’s why we have Amex Flex, our enterprise working model that provides greater flexibility to colleagues while ensuring we preserve the important aspects of our unique in-person culture. Depending on role and business needs, colleagues will either work onsite, in a hybrid model (combination of in-office and virtual days) or fully virtually.

Be part of the exciting transformation world and join a team of creative, curious and passionate professionals whose mission is to become the future of American Express by staying on the cutting edge and bringing innovation to life! The Campaign Execution Platforms team leads the technology strategy and engineering delivery of marketing campaigns.

This position works closely with Enterprise Digital & Analytics Product teams to craft, develop and support critical components and platforms for the majority of American Express products and services. Director will manage a global team (US and India) supporting the Big Data capabilities and enterprise user community. These capabilities consist of a Data Ingestion Suite providing complete, robust, enterprise-grade framework for the ingestion of data from a variety of internal and external data sources into Big Data, including SORs and derived data. Data Transformation Engineering is enabled via an interactive tool allowing business users to self-manage policies and processes while leveraging the Big Data content.

Key functions of the team include:

• Batch Data Ingestion – DI Framework supporting end-to-end data ingestion:
• System of Record source system certification
• CDM (Centralized Data Management) – Metadata repository with APIs containing detailed metadata, data lineage, data transformation, etc.)
• Metadata registration utilizing MDR (UI for Metadata Registration Portal)
• Data Standardization (column names, data types, PII tagging, etc.)
• Data Certification
• Event Engine Scheduling and workload management tool (scheduling, executing and monitoring ingestions)
• Data Writer support for derived data feeds
• Data Transformation Tool’s Intuitive UI which is used to bring SOR data in alignment with Big Data standards by taking rules to the data instead of data to the rules.

 We are looking for a dynamic, innovative, action-and-results oriented leader to lead this fast-evolving eco system.

The ideal candidate is required to be accomplished in the development and deployment of complex software solutions, with a strong focus on, Big Data (Hadoop/MapR) engineering, various Java technologies including Spring, API design and user experience. The candidate should be able to come up with innovative solutions, have a pragmatic approach to application architecture, and be excited to work on a varied tech stack. You will be expected to have a deep understanding of the functional platforms, plan, manage, and execute the tech advancement roadmap. You are required to have a futuristic mindset, ability to question status quo and technologically advance the platforms.

This is an opportunity to work in a fast-paced, Scaled Agile environment and work with a team of engineers, product owners and several technology stakeholders. You should be familiar with modern software development methodologies and be able to dive deep and rapidly iterate on ideas despite ambiguity.

Job Responsibilities:

• Bring a strong technical foundation; able to engage the team on coding practices, architecture, design, get under the hood of complex integrated architectures, coding systems, and interface design
• Collaborate successfully with product owners, designers, and a broad set of internal Technical partners (across multiple internal groups)
• Ensure product releases are high quality, deliver excellent user experiences, perform seamlessly at scale, and comply with regulatory requirements.
• Partner with peers in technologies to identify opportunities for code sharing, common services, joint development, etc.
• Keep up with the latest industry research and emerging technologies to ensure we are appropriately leveraging new techniques and capabilities.
• Demonstrate accountability while leading people with passion, enthusiasm, loyalty and integrity. Own and lead HR processes such as performance reviews, talent development, etc.
• Flawless interpersonal skills and ability to partner with executive leadership to push Technical solutions forward.
• Recruit top talent with technical skills, growth potential, design sensibility, and emotional intelligence.
• Lead teams in iterative product development using lean principles.
• Build and lead a high-performing, self-managing team of Software Engineers.
• Passion, energy, personal accountability, and a desire to lead others to excellence.
• Build and leverage strong relationships with business and Technology partners.
• Drive strategic change in tools and process.
• Deep appreciation for the importance of elegant solutions to Technical challenges.
• Consistently question assumptions, challenge the status quo, and strive for improvement.
• Never settle for being “good enough”.

• Extensive background in Software Engineering and Computer Science.
• 10+ years of demonstrable experience building and shipping large scale software products.
• Strong Experience in Agile, Agile development, code review, Hadoop, Hive, Spark, Java, Spring framework, ETL, SQL, Relational database, Cloud computing, UNIX and Web Services.
• Moderate or higher skill in SOA, Restful, Asynchronous and Microservices technologies for large scale environments
• Good knowledge of GITHub.
• Experience in Cloud technology.
• Experience with UNIX based systems including the ability to create and maintain Shell script.
• Strong understanding and experienced in CICD and DevOps
• The ability to communicate complex technical concepts clearly and concisely to non-technical individuals
• Excellent verbal and written communications skills required, with ability to create and deliver effective presentations to executive leadership and technical architects across the organization is a plus.
• Hands on experience with technology planning and road mapping
• Proven track record to lead problem analysis and resolution
• Bachelor’s Degree in Computer Science, Computer Engineering or Information Systems degree or equivalent work experience; Advanced Degree preferred