Big Data Engineer II

Key Skills

Java Programming, Python Programming, C++, C Programming, SQL, Apache Hadoop, Scala Programming, Machine learning techniques, Data science techniques, PyTorch, TensorFlow, MapReduce, R Programming

Job Description

We are seeking a passionate and intellectually curious Big Data Engineer II for our Data Engineering team. Data Engineering team is responsible for creating data pipelines in big data space including data lake and data warehouse in AWS (Amazon Web Services), Azure cloud and on premise environments.

Job Description

Essential Responsibilities:

• Design, implement, and test major subsystems of AWS or AZURE cloud platform and core service offerings using the Scrum agile framework
• Develop and follow best practices relative to design, implementation, and testing
• Prototype new ideas or technologies to prove efficacy and usefulness in production
• Build a service structure on AWS or AZURE capable of being deployed and scaled to run a variety of platform components dynamically
• Build a next-generation tools platform for creating, managing and deploying multi-channel outreach campaigns in the AWS or AZURE cloud
• Construct a state-of-the-art data lake using Hadoop or Cassandra, Apache Spark, NiFi, and Kafka
• Design & develop data pipelines for batch & streaming data sets using ETL and Data Integration tools, open source, AWS & Azure tech stack
• Mentor junior team members

Non-Essential Responsibilities:

• Other duties as assigned

Job Description

Knowledge, Skills and Abilities:

• Command-level knowledge of Java and Python programming, and the fundamentals of computer science, data structures and programming
• Experience in Big Data technologies (Hadoop, Spark, NiFi, Kafka)
• Ability to demonstrate experience in distributed UNIX environments.
• Experience in writing shell scripts
• Ability to demonstrate proficiency in Microsoft Access, Excel, Word, PowerPoint and Visio.
• Ability to multi-task and work under pressure.
• Ability to be careful and thorough with detail.
• Ability to work both independently and in a collaborative environment.
• Ability to analyze information and use logic to address work related issues and problems.
• Experience in the Healthcare Industry is a plus.

Qualifications

Minimum Education:

• The knowledge typically acquired during the course of attaining a Bachelor’s degree in Computer Science, Mathematics, or related discipline is required. A combination of education and experience may be used in lieu of a diploma.

Minimum Related Work Experience:

• 2-4 years’ experience designing and delivering production software
• 2 years’ experience designing and implementing big data high performance operational systems
• Proven experience using the Microsoft development tools and stack, e.g., TFS, Github, Eclipse, JVM, etc