Data Engineer

Key Skills

Agile methodologies, Apache Flink, Apache Kafka, AWS, Data Engineering, Data pipelines, SPARK Programming, Spark-SQL, SQL

Job Description

We're hiring a Data Engineer, reporting to the Data Engineering Manager based in Sydney. Here, you'll enable a world-class engineering practice, drive the approach with which we use data, develop backend systems and data models to serve the needs of insights, and help build Atlassian's data-driven culture. You love thinking about the ways the business can consume data and then figuring out how to build it.

• You'll partner with the data analytics and data scientist team to build the data solutions that allow them to obtain more insights from our data and use that to support important business decisions.

• You'll work with different stakeholders to understand their needs and architect/build the data models, data acquisition/ingestion processes and data applications to address those requirements.

• You'll add new sources, code business rules, and produce new metrics that support the product analysts and data scientist.

• You'll be the data domain expert who understand all the nitty-gritty of our products.

• You'll own a problem end-to-end. Requirements could be vague, and iterations will be rapid

• You'll improve data quality by using & improving internal tools/frameworks to automatically detect DQ issues.

• A BS in Computer Science or equivalent experience with 3+ years professional experience as a Data Engineer or in a similar role.

• Strong programming skills using Python

• Working knowledge of relational databases and query authoring (SQL).

• Experience designing data models for optimal storage and retrieval to meet product and business requirements.

• Experience building scalable data pipelines using Spark (SparkSQL) with Airflow scheduler/executor framework or similar scheduling tools.

• Experience working with AWS data services or similar Apache projects (Spark, Flink, Hive, and Kafka).

• Understanding of Data Engineering tools/frameworks and standards to improve the productivity and quality of output for Data Engineers across the team.

• Well versed in modern software development practices (Agile, TDD, CICD)