Staff Data Engineer

Key Skills

Agile methodologies, Analytical and Problem solving, Apache Hadoop, Apache Hive, API, Business Intelligence, Cloud computing, Cloudera, Continuous Integration & Continuous Delivery - CI/CD, DevOps, Machine learning techniques, Project management, PySpark, Python Programming, Scala Programming, SQL, Tableau

Job Description

Staff Data Engineer role for Asia Pacific Region is based out of Bangalore. 

We are looking for an expert with deep expertise in big data/data warehousing and who has experience in working with clients in implementation of data engineering pipelines, data exchange mechanisms and data science models.

The role will need design and architecture skills to advice VISA’s clients on best approaches for data engineering and data science model implementations.

The candidate will also play critical role in enabling the internal data platforms using which Data Scientists, Analysts, and BI Users drive solutions for Visa clients. The candidate is expected to act as a bridge between end-users in Asia and Visa Technology colleagues in San Francisco, influencing the development of our global data platforms whilst provisioning local tools and technologies as required. 

This is a hands-on role, and the candidate is expected to work hands on in areas such as Hadoop, Spark, Python, Tableau, Unix scripting.

We are looking for a talented, technical, proactive, energetic, and passionate person who embraces challenges and is a proven problem solver.

Key Responsibilities:

• Provide advisory and implementation support to VISA’s clients for deployment of data engineering pipelines and data science models.
• Work with VISA data scientists for optimization and automation of data science models.
• Support data exchange channels integration (Such as APIs, Secured File Transfer, other custom data exchange capabilities provided by VISA) between VISA and clients.
• Create and maintain optimal data pipeline architecture(s), based on our Global Technology Stack
• Provide leadership and support in identifying, implementing, and managing new data tools and processes, relevant to data science – on premise and cloud.
• Partner with Technology on quarterly planning cycle and support management with relevant metrics to evaluate performance, stability, and reliability of various tools.
• Identify, design, and implement internal process improvements to provide greater scalability to our existing client solutions.
• Develop custom-built packages to support the needs of Data Scientists across the region.
• Continuous focus on improving Infrastructure efficiency by analyzing logs of queries, tuning settings, translating queries if required.
• Review scripts and educate users by building training assets for beginner and intermediate level users.
• Policy guidance and administration of tools used by data scientists including git, ETL schedulers, Spark, Tableau etc. to ensure optimum utilization and efficiency

This is a hybrid position. Hybrid employees can alternate time between both remote and office. Employees in hybrid roles are expected to work from the office 2-3 set days a week (determined by leadership/site), with a general guidepost of being in the office 50% or more of the time based on business needs.

Professional Experience

• 8 - 12 years of relevant experience
• Deep knowledge of distributed data architecture, commonly used BI tools, and approaches/packages used in machine learning build
• Expertise in Design and architecture of big data platforms, data science and visualization platforms.
• Expertise in creating production software/systems using Python and/or Scala, and a proven track record of identifying and resolving performance bottlenecks in production systems.
• Experience with complex, high volume, multi-dimensional data, as well as machine learning models based on unstructured, structured, and streaming datasets.
• Good appreciation of machine learning algorithms, feature engineering, validation, prediction, recommendation, and measurement.
• Experience of working in multiple large projects with diverse cross-functional teams.
• Ability to learn new tools and paradigms as data science continues to evolve at Visa and elsewhere.
• Understanding of the Payments and Banking Industry including aspects such as consumer credit, consumer debit, prepaid, small business, commercial, co-branded and merchant
• Demonstrated ability to incorporate new techniques to solve business problems
• Demonstrated resource planning and delivery skills
• Good communication and presentation skills with ability to interact with different cross-functional team members at varying level

Technical Expertise:

• Certification in Hadoop (Cloudera or Hortonworks) and Apache Spark preferred.
• Expertise in dashboard and report development in Tableau
• Experience in coming up with data flow patterns and deployment architecture for data engineering pipeline and data science models
• Experience with APIs, container-based software deployments
• Experience in developing production systems incorporating best-in-class software engineering practices such as DevOps, Agile development, CI/CD, and scheduling technologies
• Working knowledge of Hadoop ecosystem and associated technologies such as Hive, Apache Spark, PySpark, MLlib, GraphX
• Advanced experience in writing and optimizing efficient SQL queries and Python scripts, Scala experience will be an added advantage
• Good appreciation of cloud-based technologies and data platforms
• Very strong people/technology project management skills and experience

Business and Leadership competencies:

• Results-oriented with strong problem-solving skills and demonstrated intellectual and analytical rigor
• Good business acumen with a track record in solving business problems through data-driven quantitative methodologies. Experience in payment, retail banking, or retail merchant industries is preferred
• Team oriented, collaborative, diplomatic, and flexible
• Detailed oriented to ensure highest level of quality/rigor in reports and data analysis
• Proven skills in translating analytics output to actionable recommendations and delivery
• Experience in presenting ideas and analysis to stakeholders whilst tailoring data-driven results to various audience levels
• Exhibits intellectual curiosity and a desire for continuous learning
• Demonstrates integrity, maturity, and a constructive approach to business challenges
• Role model for the organization and implementing core Visa Values
• Respect for the Individuals at all levels in the workplace
• Strive for Excellence and extraordinary results
• Use sound insights and judgments to make informed decisions in line with business strategy and needs
• Ability to influence senior management within and outside Analytics groups
• Ability to successfully persuade/influence internal stakeholders for building best-in-class solutions