Cloud Data Engineer

Key Skills

Aartificial intelligence, AWS, Azure, Bash scripting, Big Data Technology, Business Intelligence, Continuous Integration & Continuous Delivery - CI/CD, Data Analytics, Data Warehousing, Design, DevOps, Docker, ETL frameworks, GitLab, Google Cloud Platform (GCP), Java Programming, Jenkins, Kubernetes-K8s, Looker, Machine learning techniques, NoSQL, PowerBI, Python Programming, QlikView, Scala Programming, SQL

Job Description

Overview

Nortal is looking for an experienced Cloud Data Engineer for our client –a professional services organization based in Vancouver, specializing in the convergence of business strategy and technological implementations. Client’s core expertise lies in Big Data, Cloud Technology, and Analytics, including Advanced Analytics in AI and machine learning.

Client works with mid-market and large enterprises across North America, devising and scaling Data & Analytics strategies and the main focus involves connecting disparate operating systems, consolidating them in the cloud, harmonizing them through modeling, and operationalizing them to drive substantial business impact.

As a Data Engineer within client’s team, your role will be pivotal in assisting clients with the automation of real-time and batch data ingestion pipelines, analytics, and the implementation of best-in-class practices for data warehousing and data lakes. You will collaborate closely with cross-functional teams, including cloud experts, advanced analytics professionals, and digital solution specialists, to tackle cutting-edge data challenges and capitalize on opportunities.

Responsibilities

• Design and implement data ingestion pipelines for batch or real-time streaming processes feeding into data warehouses and data lakes
• Identify, design, and implement process improvements, including automation of manual processes, enhancing data delivery efficiency, and optimizing infrastructure scalability
• Provide testing, monitoring, debugging, and technical documentation throughout the data lifecycle and development stages
• Deploy, operate, and enhance CI/CD pipelines for efficient web and cloud-based application delivery
• Configure and monitor cloud-based services to ensure high availability, performance, security, and adherence to client technical requirements and service level agreements
• Monitor data-related performance, troubleshoot errors, and resolve issues
• Document standards and best practices for critical data processes and architecture
• Share insights on emerging client data requirements and contribute to Fastloop's competitive positioning and value proposition
• Stay updated on new technologies, processes, and products that can benefit both our team and clients
• Collaborate between Technical and Business teams as needed to contribute to overall project success
• Additional responsibilities specific to candidates and experience may be discussed on a project basis

Qualifications

• 3+ years of relevant experience in data engineering, data ingestion, and/or data warehousing
• 2+ years of experience in a software engineering environment
• Demonstrated proficiency in at least one major cloud technology (GCP/Azure/AWS) with a focus on big data
• Experience in DevOps practices and CI/CD pipelines within a production context
• Proven track record in implementing cloud monitoring strategies
• Proficiency in agile or lean development methodologies
• Strong analytical and problem-solving skills, capable of handling complex challenges
• Degree or technical diploma in Computer Science/Engineering or a related field
• Familiarity with SQL and NoSQL systems
• Proficiency in some programming languages such as Scala, Java, Python, or Bash.
• Exposure to automated testing systems
• Understanding of data modeling, ETL processes, data warehousing, and business intelligence reporting tools
• Familiarity with CI/CD, containerization, and virtualization tools like Gitlab, Jenkins, Kubernetes, Docker
• Experience with data modeling and design using BI platforms like Power BI, Qlik, or Looker