Data Engineer - X Delivery

Key Skills

Apache Airflow, Apache Cassandra, API, AWS, Azure, Big Data Technology, Continuous Integration & Continuous Delivery - CI/CD, Data Engineering, Data science techniques, Design, DevOps, Github, Google Cloud Platform (GCP), HBase, Jenkins, Machine learning techniques, MongoDB, NoSQL

Job Description

As a part of BCG’s X team, you will work closely with consulting teams on a diverse range of advanced analytics and engineering topics. You will have the opportunity to leverage analytical methodologies to deliver value to BCG's Consulting (case) teams and Practice Areas (domain) through providing analytical and engineering subject matter expertise.

As a Data Engineer, you will play a crucial role in designing, developing, and maintaining data pipelines, systems, and solutions that empower our clients to make informed business decisions. You will collaborate closely with cross-functional teams, including data scientists, analysts, and business stakeholders, to deliver high-quality data solutions that meet our clients' needs.

YOU'RE GOOD AT

• Delivering original analysis and insights to case teams, typically owning all or part of an analytics module whilst integrating with a case team.
• Design, develop, and maintain efficient and robust data pipelines for extracting, transforming, and loading data from various sources to data warehouses, data lakes, and other storage solutions.
• Building data-intensive solutions that are highly available, scalable, reliable, secure, and cost-effective using programming languages like Python and PySpark.
• Deep knowledge of Big Data querying and analysis tools, such as PySpark, Hive, Snowflake and Databricks.
• Broad expertise in at least one Cloud platform like AWS/GCP/Azure.
• Working knowledge of automation and deployment tools such as Airflow, Jenkins, GitHub Actions, etc., as well as infrastructure-as-code technologies like Terraform and CloudFormation.
• Good understanding of DevOps, CI/CD pipelines, orchestration, and containerization tools like Docker and Kubernetes.
• Basic understanding on Machine Learning methodologies and pipelines.
• Communicating analytical insights through sophisticated synthesis and packaging of results (including PPT slides and charts) with consultants, collecting, synthesizing, analyzing case team learning & inputs into new best practices and methodologies.

Communication Skills:

• Strong communication skills, enabling effective collaboration with both technical and non-technical team members.

Thinking Analytically

• You should be strong in analytical solutioning with hands on experience in advanced analytics delivery, through the entire life cycle of analytics. Strong analytics skills with the ability to develop and codify knowledge and provide analytical advice where required.

What You'll Bring (Experience & Qualifications)

• Bachelor's / Master's degree in computer science engineering/technology
• At least 2-4 years within relevant domain of Data Engineering across industries and work experience providing analytics solutions in a commercial setting.
• Consulting experience will be considered a plus.
• Proficient understanding of distributed computing principles including management of Spark clusters, with all included services – various implementations of Spark preferred.
• Basic hands-on experience with Data engineering tasks like productizing data pipelines, building CI/CD pipeline, code orchestration using tools like Airflow, DevOps etc.

Good to have:-

• Software engineering concepts and best practices, like API design and development, testing frameworks, packaging etc.
• Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
• Knowledge on web development technologies.
• Understanding of different stages of machine learning system design and development.

YOU'LL WORK WITH

• You will work with the case team and/or client technical POCs and border X team.