Senior Data Platform Engineer

Key Skills

Amazon Simple Storage Service (S3), Apache Airflow, Apache Kafka, Big Data Technology, Data Warehousing, Databricks, Docker, ETL frameworks, Grafana, JavaScript, Kubernetes-K8s, Prometheus, Python Programming, Scala Programming, Snowflake, Unix Operating system

Job Description

This is an opportunity within the Data & Analytics team at RBC Global Asset Management. We are seeking a hands-on, highly skilled, and motivated individual to join our dynamic team as a Senior Data Platform Engineer. As a key member of our data engineering team, you will play a crucial role in the development and implementation of GAM’s Data Platform. Your primary focus will be on designing, implementing, and maintaining robust data infrastructure to support our organization’s data needs and advance our capabilities pertaining to data engineering, BI, Machine Learning and AI.

What you will do:

• Work closely with Data Engineering and Analytics team to drive business decisions using data.

• Build scalable data platform infrastructure using cloud native technologies to enable Data Engineering, Data Science and Reporting.

• Take ownership of end-to-end deliverables and ensure high quality data solutions adhering to data platform architecture guidelines and engineering best practices.

• Enable the deployment of batch and event-driven data pipelines using latest technologies such as Spark, NiFi, Kafka, and Airflow.

• Configure and manage scalable and managed SQL based self-serve compute platform using Starburst Trino.

• Implement data management tools to streamline data processing, testing, preparing, deploying, and monitoring.

• Improve the speed and quality of the data development process by applying DevOps principles to data workflow.

• Ensure data security standards are applied across the data pipelines.

• Troubleshoot and resolve data integration and data processing issues.

• Perform proof-of-concepts to implement hybrid cloud data strategy involving on-prem systems like S3, OpenShift, Trino, Spark with Cloud based solutions like Databricks, Azure, Snowflake, etc.

• Apply design-thinking and agile mindset in working with other engineers and business stakeholders to continuously experiment, iterate and deliver on new initiatives.

• Stay informed about emerging technologies and trends in data engineering domain.

• Lead, mentor, and inspire a team of engineers to achieve high performance levels.

What you need:

• 3-5 years of hands-on experience in big data processing using Spark, Python, NiFi, Kafka, and Airflow for on-premises and Cloud data platforms (Snowflakes, Databricks, Azure, AWS, GCP)

• 2-3 years of experience with solution design, data ingestion, scheduling & monitoring of ETL/ELT jobs.

• 2-3 years of system administration including Docker and Kubernetes platforms.

• Strong knowledge of UNIX-like operating systems and shell scripting skills.

• Expertise with DevOps tools, automation, and using Data Ops to develop data flows and the continuous use of data.

• Proficiency in writing and optimizing SQL queries and at least one programming languages like Java, Scala and/or Python.

Nice-to-have

• Experience with OpenShift, AirFlow, S3, Trino, Ranger and Hive.

• Knowledge of networking, information security and vulnerability management.

• Knowledge of creating dashboards using Prometheus and Grafana.

• Knowledge about data science tools and libraries.

What’s in it for you?

We thrive on the challenge of being our best, thinking progressively, continuing to grow, and working together to deliver trusted advice to help our clients thrive, and our communities prosper. We care about each other, reaching our potential, making a difference in our communities, and achieving mutual success.

• A comprehensive Total Rewards Program, including bonuses, flexible benefits, and competitive compensation.

• Leaders who support your development through coaching and managing opportunities

• Opportunities to take on progressively greater accountabilities

• Work in a dynamic, collaborative, progressive, and high-performing team