Staff Data Engineer Data Science/Data Engineering

Key Skills

Agile methodologies, Algorithms, Analytical and Problem solving, Apache Hadoop, Apache Hive, API, Business Intelligence, Cloud computing, Cloudera, Continuous Integration & Continuous Delivery - CI/CD, DevOps, ETL frameworks, Google Dataflow, JIRA, Machine learning techniques, MLOps tools, Pandas, Project management, PySpark, Python Programming, Scala Programming, Scikit-learn, SQL, Tableau

Job Description

Staff Data Engineer role for Asia Pacific Region is based out of Bangalore. We are looking for an individual with deep expertise in big data/data warehousing and can build large-scale data processing systems by using the latest database and data processing technologies. The role will work as part of the project team to support the design, development, and delivery of data engineering solutions to the market data science teams and to VISA clients. He/she will take responsibility for building and running data pipelines, designing our local data warehouse, data marts and data frameworks, and catering for different data presentation techniques based on requirements. Also, the role provides as a bridge between our local end-users and our Visa Technology colleagues in San Francisco, influencing the development of our global data platforms whilst provisioning local tools and technologies as required. 

Principal Responsibilities

• Work as part of the project team to support the design, development, and delivery of data engineering solutions to the market data science teams and to VISA clients adhering to scope and timelines.
• Create and maintain optimal data pipelines, data marts and architecture(s), based on our Global Technology Stack
• Collaborate with the global data engineering team to adopt global engineering standards in AP region.
• Develop custom-built packages to support the needs of Data Scientists across the region
• Identify, design, and implement internal process improvements to provide greater scalability to our existing client solutions
• Work with broader business stakeholders to assist clients and consultants with their data and infrastructure needs
• Partner with Technology on quarterly planning cycle and support management with relevant metrics to evaluate performance, stability, and reliability of various tools.
• Continuous focus on improving Infrastructure efficiency by analyzing logs of queries, tuning settings, translating queries if required.
• Provide technical and business support for development of new platforms, tools for data science using both on-prem and cloud technologies.
• Review scripts for best practices, educating user base and building training assets for beginner and intermediate users.
• Policy guidance and administration of tools used by data scientists including git, ETL schedulers, Spark, Tableau etc. to ensure optimum utilization and efficiency.

Qualifications

Basic Qualifications and experience

• 8+ years of data engineering and support experience.
• Deep knowledge of distributed data architecture, commonly used BI tools, and approaches/packages used in machine learning build
• Expertise in creating production software/systems in Python and/or Scala, and a proven track record of identifying and resolving performance bottlenecks in production systems.
• Experience with complex, high volume, multi-dimensional data, as well as machine learning models based on unstructured, structured, and streaming datasets.
• Good appreciation of data science modeling, machine learning algorithms and feature engineering
• Experience in planning, organizing, and managing multiple large projects with diverse cross-functional teams.
• Ability to learn new tools and paradigms as data science continues to evolve at Visa and elsewhere.
• Understanding of the Payments and Banking Industry will be a plus
• Demonstrated ability to incorporate new techniques to solve business problems
• Demonstrated resource planning and delivery skills
• Good communication and presentation skills with ability to interact with different cross-functional team members at varying level

Technical Expertise

• Certification in Hadoop (Cloudera or Hortonworks) and Apache Spark preferred. 
• Advanced experience in writing and optimizing efficient SQL queries and Python/PySpark scripts; Scala experience will be an added advantage.
• Expertise in dashboard and report development using visualization tools such as Tableau, MicroStrategy
• Good understanding of Agile way of working, tools such as JIRA and practical experience of working in Agile teams.
• Experience in developing production systems incorporating best-in-class software engineering practices such as DevOps, CI/CD, and automation technologies
• Experience in deployment, operationalization of Machine Learning models using MLOps techniques
• Working knowledge of Hadoop ecosystem and associated technologies, e.g., Hive, Apache Spark, MLlib, GraphX, sci-kit, and Pandas
• Experience with APIs, containerized deployments will be an added advantage
• Good appreciation of cloud-based technologies and data platforms

Additional Information

Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.