Big Data Engineer

Key Skills

Apache Hadoop, C Programming, C++, Data science techniques, Java Programming, Machine learning techniques, MapReduce, Python Programming, PyTorch, R Programming, Scala Programming, SQL, TensorFlow

Job Description

As a Big Data Engineer, you will assist leadership for projects related to Big Data technologies and software development support for client projects. You will help with migrating our current Big Data clusters to the latest releases, and implement High-Availability and High-performance features. You will help with deploying, optimizing and supporting new Big Data clusters on the Public Cloud.

What you will do:

• Responsible for implementation and ongoing administration of Hadoop infrastructure in Cloudera
• Deploy, manage, maintain, and optimize Big-Data deployments to Public Cloud using Databricks, Qubole etc.
• Optimize query performance and tuning for jobs running on Spark, Impala, and Hive
• Automate repetitive tasks by performing hands-on programming with Perl, Bash, and/or Python
• Provide Operational Support and maximize Service availability by monitoring, troubleshooting, and resolving issues

What you will need:

• Highly motivated to learn new technologies, with strong Engineering mindset
• Quick learner with strong aptitude for solving problems, writing code, and building Automation
• Strong team player and willing to collaborate closely with others to get projects done
• 2+ years of Big Data Administration experience
• 2+ years of hands-on Scripting skills in Perl, Python, Bash Shell
• Knowledge of Hadoop based data manipulation/storage technologies such as HDFS, MapReduce, Spark
• Experience in capacity planning, cluster crafting and deployment, troubleshooting and performance tuning
• Good knowledge of Linux OS and Database SQL
• Excellent verbal and written communication skills
• Bachelor's degree in related field of study, or equivalent experience

What will set you apart?

• Experience in Data Engineering and Data Sciences
• Experience in Big Data deployments in Public Cloud such as Databricks, Qubole
• Excellent working knowledge of Linux Systems on VMWare
• Automation using Ansible #GD17