Hadoop/NiFi Administrator

Key Skills

3D data processing, Aartificial intelligence, Apache Hadoop, Apache NiFi, AWS, Azure, Big Data Technology, Data Engineering, Data pipelines, Data science techniques, Docker, ETL frameworks, HDFS, HIPAA, Kubernetes-K8s, Linux Operating system, Machine learning techniques, MapReduce, Scrum Agile Methodology, Unix Operating system

Job Description

The Hadoop/NiFi Administrator will play a pivotal role in managing and overseeing Gainwell Technologies' Hadoop and NiFi clusters. As part of our dynamic technology team, this individual will be responsible for ensuring the robustness, scalability, and security of our data infrastructure. The role demands a deep technical expertise, a passion for problem-solving, and a commitment to continuous improvement. Working closely with various departments, the Hadoop/NiFi Administrator will contribute to strategic planning, technology evaluations, and alignment with overall business goals. This position offers an opportunity to thrive in a fast-paced environment where innovation and collaboration are highly valued. 

Your role in our mission

Essential Job Functions

• Install, configure, and maintain Hadoop and NiFi clusters, ensuring high availability and fault tolerance. 
• Collaborate with data engineers and data scientists to optimize data pipelines, workflows, and data processing tasks. 
• Monitor system performance and capacity, conducting proactive tuning and scaling as needed. 
• Implement and enforce security protocols, access controls, and data governance policies in accordance with industry best practices. 
• Conduct regular backups, disaster recovery simulations, system upgrades, and patches. 
• Provide expert technical support, troubleshooting, and guidance to internal teams. 
• Develop and maintain comprehensive documentation, including architecture diagrams, configuration guidelines, and operational procedures. 
• Evaluate and recommend new technologies, tools, and best practices to enhance the existing infrastructure. 
• Coordinate with vendors and external support teams when necessary. 
• Assist with budgeting, planning, and reporting related to the Hadoop and NiFi environment. 

What we're looking for

• 5+ years of hands-on experience as a Hadoop/NiFi Administrator or similar role in a large-scale environment. 
• Deep understanding of the Hadoop ecosystem components (HDFS, YARN, MapReduce, etc.) and Apache NiFi. 
• Experience with various Hadoop distributions (e.g., Cloudera, Hortonworks) and integrating with other data platforms. 
• Proficiency in Linux/Unix systems, shell scripting, Java, and other programming languages. 
• Strong knowledge of networking, virtualization, cloud services (AWS, Azure, etc.), and containerization technologies (Docker, Kubernetes). 
• Familiarity with monitoring, logging, and alerting tools such as Prometheus, Grafana, or similar. 
• Strong communication skills, with the ability to convey complex technical concepts to non-technical stakeholders. 
• Relevant certifications in Hadoop, NiFi, or related technologies are a plus. 
• Proven track record of successfully implementing and managing complex projects. 

What you should expect in this role

• Remote work

Competency1

• Master's degree in Computer Science, Engineering, or a related field preferred. 
• Certifications in Hadoop, NiFi, or other related big data technologies. 
• Extensive experience in managing multi-cluster environments and cross-platform integrations. 
• Familiarity with data warehousing, ETL processes, and real-time data processing. 
• Knowledge of data privacy regulations and compliance requirements such as GDPR, HIPAA, etc. 
• Proven ability to mentor and lead technical teams, fostering a culture of collaboration and continuous learning. 
• Experience in project management methodologies (e.g., Agile, Scrum) and utilizing project management tools. 
• Exposure to Artificial Intelligence (AI) and Machine Learning (ML) technologies is a plus. 
• Strong interpersonal skills, with a proven ability to build relationships and effectively communicate with stakeholders at all levels. 
• Demonstrated commitment to staying up to date with emerging technologies and industry trends. 

The pay range for this position is $71,100.00 - $101,600.00 per year, however, the base pay offered may vary depending on geographic region, internal equity, job-related knowledge, skills, and experience among other factors. Put your passion to work at Gainwell. You’ll have the opportunity to grow your career in a company that values work flexibility, learning, and career development. All salaried, full-time candidates are eligible for our generous, flexible vacation policy, a 401(k) employer match, comprehensive health benefits, and educational assistance. We also have a variety of leadership and technical development academies to help build your skills and capabilities.