Software Development Engineer - Big Data - DevOps role

Key Skills

Java Programming, Python Programming, C++, C Programming, SQL, Apache Hadoop, Scala Programming, Machine learning techniques, Data science techniques, PyTorch, TensorFlow, MapReduce, R Programming

Job Description

Amadeus is currently building Big Data, Analytics and Machine-learning based applications. This is done using a Big Data Platform hosted in Azure and relying on the most advanced Data and ML managed services from Microsoft and third-party vendors (Snowflake, Databricks).

These services shall be provided as a fully integrated platform, providing nice user-experience for the data engineers and scientists, and an easy sharing and secured access to the Amadeus data assets across the Data Mesh. While being developed by a tremendous team with many profiles, the platform security, reliability and serviceability is under the responsibility of Business Intelligent Platform department.

Summary of the role

The BIP (Business Intelligent Platform) department is playing a key role in Amadeus to provide

adapted, reliable, secure and efficient data storage and data processing solutions for Amadeus analytic

applications.

The Big Data platform is at the early stages of development and receives constant feedback for improvement and enhancement from a growing number of early adopters. In addition to the innovative data-science applications, the platform is also designed to host all the existing data and workloads moving from the legacy Hadoop based infrastructures.

DevOps activities or data science capabilities. The Big Data platform will become integral part of Amadeus Cloud ecosystem and shall be deployed to the public Cloud with respect to the Cloud Design authority recommendations.

You main responsibilities

• Handle migration of Hadoop workloads to Azure Databricks
• Handle migration of the Hadoop Datalakes to the Azure Data Mesh
• Handle migration of the MPP query engines to Snowflake Datawarehouses
• Deploy open-source services via Kubernetes Helm charts
• Develop ad-hoc services with serverless low-code application frameworks
• Integrate analytics with data-visualization tools
• Develop Biceps and ARM templates running in Azure Devops
• Develop frameworks to support the full lifecycle of Machine learning applications
• Full automation of typical application and platform management activities: Platform deployment, stack upgrades, security integration, and all typical cluster maintenance activities.
• Operation excellence, and expert support to operational and R&D teams
• Providing guidelines and support to implement efficient Big data pipelines : Delivering expertise and guidance to data engineers and data scientists is key to the success of the different projects. In addition, it contributes to manage efficiently the resources in order to minimize the hardware cost.
• Building expertise and anticipating evolution of Cloud technologies
• Contribute with applicative developers to define requirements for development of new or improvement of existing solutions.
• Propose viable technical solutions to applicative developers for validation (e.g. non-regression test framework).
• Define Amadeus Standards for Big Data platforms extending current Amadeus Standards.
• Support the end user in the Production phase by debugging existing software solutions, or investigating stability or performance issues
• Design technical solutions and perform feasibility studies associated to future standards
• Conduct unit, integration and performance tests of the software and ensure a level of quality in line with the Amadeus guidelines.
• Participate in the validation / acceptance phase of the product cycle ensuring the fine-tuning necessary to finalize the product.
• Produce software documentation /architecture presentation.

About the ideal candidate

• Cloud (Azure services), PaaS (Amadeus Cloud Services, Kubernetes...)
• Programming languages: Scala, Python, R, shell
• Spark and its respective eco-system (SparkML, Spark Streaming, Deltalake)
• ML Ops (ML model management, online and offline prediction services, MLFlow)
• SQL, Datawarehouses, Data modeling, Data Management, Business intelligence
• Infrastructure as Code ; Automation (Ansible, Terraform...)
• Experience with Big Data infrastuctures (Hadoop) at scale is a plus
• A passion for new technologies and their valorisation in an industrial context
• Agile/Scrum, ITIL and security knowledge

What we can offer you

• competitive remuneration compared to the market practices and coherent with the company results
• individual and company bonus?
• 6 weeks holiday, plus pension contribution and healthcare insurance
• Flexible working conditions?
• Professional development in a truly international and multisite environment with a great mix of people
• Experience in an environment with unique complexity and a hardly matched criticality among the leading tech companies
• A large selection of learning opportunities available to broaden your knowledge and enhance your soft skills

Work from anywhere

• Flexible teleworking from 1 day per week to full weeks to full-remote
• Home office monthly alowance + setting-up premium