Pyspark Data Engineer

Key Skills

Apache, Apache Hadoop, Apache Kafka, Python Programming, Scala Programming

Job Description

Pyspark Data Engineer

8 to 12 years 

Technical Skills

• Pyspark

• Spark

Nice to have skills.

• Scala

• Python

• Apache Hadoop

• Kafka Admin

Roles/Responsibilities:

• Expertise in Hadoop Admin Expertise in EMR Admin2

• Hands on experience in Cassandra administration –

• Designing for consistency Knowledge on load distribution and replication configuration

• Strong and capable of Developing efficient pig and hive scripts with joins on datasets using various techniques.

• Strong and capable of Assessing the quality of datasets for a Hadoop data lake.

• Strong and capable of Fine-tuning Hadoop applications for high performance and throughput

• Strong and capable of Troubleshooting and debug any Hadoop ecosystem run time issues.

• Able to learn and adopt to the new Hadoop Stacks

• Capable of interaction with Hadoop Client leads to obtain requirements and discuss solutions to cater to business needs.

• Coordinate with offshore and ensure delivery as per schedule.

• Strong and capable of Proposing best practices & standards.

• Advance proficiency in Java programming.

• Proficient understanding of code versioning tools such as GitHub.

Qualifications:

• Bachelor’s degree in engineering or relevant field

• 5-8 years of overall IT experience and 6+ years of experience in Hadoop Administration Cassandra Admin/Dev Oracle Golden gate Kafka admin activities

• Experience working on KSQL and Kafka Streams for data processing within Kafka.

• Develop the Microservice Key space with Microservice based tables for consumption.

• Experience in AWS Cloud technologies

• Experience in different file systems like XML JASON.

• Understanding of security and identify concepts and familiar with authentication single-sign-on directory services etc.

• Knowledge to operate with defect management tools such as JIRA QC Rally etc.

• Ability to work in sprints (agile project)

• Should have a thorough understanding of Agile framework.

• Must have good communication skills to liaise with the customer stakeholders.

Primary Location

Richardson – TX

Salary and Other Compensation:

The annual salary for this position is depending on experience and other qualifications of the successful candidate.

This position is also eligible for Cognizant’s discretionary annual incentive program, based on performance and subject to the terms of Cognizant’s applicable plans.

Benefits: Cognizant offers the following benefits for this position, subject to applicable eligibility requirements:

• Medical/Dental/Vision/Life Insurance
• Paid holidays plus Paid Time Off
• 401(k) plan and contributions
• Long-term/Short-term Disability
• Paid Parental Leave
• Employee Stock Purchase Plan