Data Engineer/Analytics Developer

Key Skills

Apache Hadoop, Apache Kafka, Continuous Integration & Continuous Delivery - CI/CD, Docker, Github, Kubernetes-K8s, NoSQL, NumPy, PowerBI, Python Programming, SPARK Programming, SQL, Tableau

Job Description

Key Tasks and Responsibilities:

• Design, develop, release and maintain reliable, secure and scalable solutions in a cloud environment (strong preference for Azure), preferably using infrastructure as code (e.g. Terraform).
• Build reliable, secure, and scalable data pipelines and production-ready pipelines that allow our engineers to bring our application/ ML models to production quickly and responsibly.
• Identify business and technical requirements to help define a robust data extraction & transformation process, collaborating with data science, product, and other technical teams.
• Develop data workflow monitoring and alerting solutions that provide diagnostic & actionable information to stakeholders.
• Ensure delivery of high-quality data engineering solutions, establishing, and following best practices for development, testing, and deployment.
• Participate in review of code and solution documentation ensuring it is according to the requirements and functional specifications.
• Use agile software development principles, DevOps, Data Engineering, and MLOps practices to facilitate a bigger data science impact.
• Work cross-functionally with Data Engineering, ML Engineering, and DevOps to resolve issues and standardise practices.
• Where appropriate, provide feedback to the Solution Architect and fellow engineers in the development of solutions.
• Highlight risks and issues that affect the development or deliveries.
• Form strong and trusted relationship with the Product Owner; ensuring that the Product Owner is fully aware of the technical capabilities and shortcomings of the product allowing for sensible technical investment and debt repayment decisions

Who we are looking for:

While reading the following list, please consider that we value attitude more than your expertise. Are you ready to be challenged on delivering an MVP that maximizes customer value? Then you should apply even if you don’t feel you fit the role 100%.

• 5+ years of data/ML engineering experience preferably in large scale projects.
• Solid knowledge of Python ecosystem for data engineering applications as evidenced by earlier work in data/ML/software engineering. Able to confidently write elegant source code with minimal supervision.
• Experience in working with SQL/NoSQL databases and can create E2E pipelines.
• Demonstrable experience designing, building, and orchestrating various software applications/ML pipelines with tools like Azure Data Factory.
• Solid Azure Cloud (ADF, Blob, Webapps, Function Apps, Cosmos DB, Data Bricks), resource provisioning (docker and Kubernetes a plus) experience.
• Experience with big data tools: Kafka, Hadoop, Spark and similar technologies. Experience with stream-processing systems.
• Comfortable working with different technologies across the software and machine learning stack for data transformation, automation (e.g. GitHub Actions) and monitoring.
• Familiar with data validation practices and standards (e.g. data testing, quality monitoring, etc).
• Knowledge of software engineering best practices across the product development lifecycle, including coding standards, code reviews, pair programming, build processes, testing, operations, and CI/CD.
• Experienced deploying containerised microservices with Docker on Kubernetes.
• Passionate about maintaining high standards when it comes to source code, documentation, system design, and production systems.
• A caring attitude towards the personal and professional development of the wider team and enthusiasm for nurturing a collaborative and dynamic culture.
• Ability to build strong relationships with stakeholders with diverse cultural backgrounds and in different geographies.

Nice to Have Skills:

• Previously created or enhanced CI/CD build and releases pipelines for Machine Learning Projects.
• Experience in working with scripting languages such as YAML, PowerShell, Terraform etc.
• Core Python skills like: Numpy, Panda, Django , PySpark.
• Experience working on Power BI, Tableau Tools.

Maersk is committed to a diverse and inclusive workplace, and we embrace different styles of thinking. Maersk is an equal opportunities employer and welcomes applicants without regard to race, colour, gender, sex, age, religion, creed, national origin, ancestry, citizenship, marital status, sexual orientation, physical or mental disability, medical condition, pregnancy or parental leave, veteran status, gender identity, genetic information, or any other characteristic protected by applicable law. We will consider qualified applicants with criminal histories in a manner consistent with all legal requirements.