Big Data Engineer - Vancouver

Key Skills

Java Programming, Python Programming, C++, C Programming, SQL, Cloud computing, Scala Programming, Machine learning techniques, Data science techniques, MATLAB Programming, PyTorch, TensorFlow, R Programming

Job Description

We are looking for an experienced Big Data Engineer to join the team and help us build world-class scalable and efficient data storage, which is to improve the overall performance of GeoComply products. If you enjoy working with large data sets, finding the best in class solutions to satisfy business requests and love tackling challenging problems, you are very welcome!

As a Big Data Engineer with GeoComply, you will help design and develop a world-class data management platform. For this position we are open to people working remotely in the US or Canada as long as you are legally entitled to work in the country you live in.

What You Will Be Doing

• Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities.
• Implementing ETL process to transform data from OLTP databases to OLAP DB and Data Lake using event streaming platforms such as Kafka.
• Develop, transform large datasets and maintain robust data pipelines that can support various use cases with high performance.
• Monitoring performance and advising any necessary infrastructure changes.
• Defining data retention, data governance policies and framework.

About You

• At least 5 years of experience in Java or Scala programming languages.
• At least 5 years of experience with Big Data, Java Spring, Kafka Streams, Spark Streams frameworks.
• Experience in large-scale deployment and performance tuning.
• Experience with schema design and dimensional data modelling.
• Experience with non-relational and relational databases (MySQL, MongoDB)
• Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
• Experience with data pipeline and workflow management tool
• Fluent in written and spoken English.
• Strong analytical and problem-solving skills.

Bonus Points if You

• Have experience with Delta Lake technology
• Good knowledge of Docker/Kubernetes is a plus
• Good knowledge of Kibana and Elasticsearch/ELK stack is a plus