Big Data Admin Specialist

Key Skills

Apache Airflow, Apache Cassandra, Apache Hive, Apache Impala, Apache Kafka, Apache NiFi, Azure COSMOS, Big Data Technology, Cloudera, Data Catalog, Design, HDFS, PostgreSQL, Python Programming, SPARK Programming, Sqoop

Job Description

The primary responsibility of the Big Data Admin Specialist will be to deliver, support and maintain a highly effective Data Lake platform, primarily based around Cloudera and Azure Big Data Cloud technologies, which underpins Mitie's strategic objective to become the industry leader in facilities management and connected workspace. A technical hands-on role with focus on delivering and providing necessary assurance over delivery from our partners and designing and building complex solutions – the successful candidate must have extensive proven experience in implementing BigData platform technical solutions and effective in both new and more established technologies across the development stack.

Main Duties:

• Implement, maintain and sustain the Big Data Platform
• Design, support and enhance the Data Lake suite including, but not limited to: Ranger, Atlas, Data Catalog, Nifi, Kafka, HDFS, Impala, Yarn, Cloudera CDP experiences, Hive, Hue, Spark, Sqoop, and Airflow
• Design, support and enhance the Azure Data Platform and Azure Managed Databases – SQL, Cosmos DB, PostgreSQL, Azure Cache for Redis along with other platforms like Apache Cassandra etc
• Perform troubleshooting, support patching, upgrades, migrations, cluster hardware replacement/new buildouts
• Provide Recommendations for workflow changes to better utilize the Cloudera tools, hardware allocation and architecting solutions related to Cloudera Components 
• Support adding hosts, user management tasks, assisting with analytical reports, optimizing and tuning, implementation and maintenance
• Design complex solutions that effectively work end-end across system components and various applications across the development stack (“Full-stack”) following best practice, coding standards and principles
• Generate new ideas for delivering value from new technology solutions using all the Data capabilities available
• Ensure development is fully documented, unit tested and peer reviewed
• You lead the resolution of technical issues arising on the project promptly to keep the project on track
• Be an advocate of quality through best-in-class development standards, lead by example
• Ensure non-functional requirements (performance, monitoring/ alerting, security, and on-going support) are considered during development; implement DevSecOps
• Guarantee overall software quality and enforce engineering standards
• Ensure there is a good balance between product innovation, technical debt reduction and delivery goals
• Take ownership and responsibility for self tasks and work with minimum governance and steer; co-ordinate with team manager, project leads and managers on project related and other activities
• Complete any ad-hoc duties as requested

Person Specification:

Essential:

• SC Cleared or eligible for SC Clearance
• Extensive hands-on Big-Data administration, development experience (5-10 years)
• Hands-on experience in Yarn, Hive, Hue, Spark, Sqoop, and Airflow, Azure Data Factory and other technologies from the Azure Big Data stack
• Extensive hands-on experience in Cloud platform technologies, mainly Microsoft Azure with knowledge of core cloud computing concepts including compute, storage, networking, databases, identity and security and higher order capabilities including serverless and IoT
• Working knowledge of various database technologies like Cosmos DB, Apache Cassandra etc
• Exposure and hands-on experience working across the technology stack (“Full-stack”)
• Strong experience of design, development and support of IT solutions
• Experience of working with with Onshore/Offshore teams
• Excellent written and verbal skills
• Logical and critical thinking skills
• Well organised, methodical and disciplined approach
• Willingness to work outside standard working hours if project timelines require

Desirable:

• Hands-on experience in Python
• Certified CCA and/or Azure Administrator would be advantageous
• Certifications and courses in any of the below would be desirable:

Microsoft Certified: Azure Fundamentals

Microsoft Certified: Azure Cosmos DB Developer Specialty