Big Data Engineer

Key Skills

Java Programming, Python Programming, C++, C Programming, Big Data Technology, Scala Programming, Keras software library, PyTorch, TensorFlow, Data Engineering, Large scale data processing

Job Description

We’re looking for a Big Data Engineer who can find out the solution to support the requirement on service operation and product development. As a Big Data Engineer, you’ll understand and manage our data, work with the engineering team and operation team to figure out the tough problem about service operation and product design.

Job Responsibilities:

• Gather and process raw data at scale.
• Process unstructured data into structured data, manage schema of new data.
• Manage data access to protect data in a safe way. 
• Read, extract, transform, stage and load data to selected tools and frameworks as required.
• Perform tasks such as writing scripts, write SQL queries, etc.
• Work closely with the operation team to advise the solution about service scalability, health monitoring and refining optimization by data analysis.
• Work closely with the engineering team to monitor product performance and track product quality.
• Analyze processed data.
• Monitoring data performance and modifying infrastructure as needed.
• Define data retention policies.

Job Requirements:

• 2+ years of recent experience in data engineering.
• Bachelor’s Degree or more in Computer Science or a related field.
• Experiences on Cloudera CDH platform or AWS Cloud Services, Spark programing, Impala SQL Language, Analyze data via Hive, etc.
• Experience in system level including Linux, and Shell scripting are required
• A solid track record of data management showing your flawless execution and attention to details.
• Strong knowledge of and experience with statistics.
• Programming experience, ideally in Python, Java or Scala, and willingness to learn new programming languages to meet goals and objectives.
• Experience in C, JavaScript or other programming languages is a plus.
• Knowledge of ETL, visualization and reporting, with an understanding of the best, most efficient use of associated tools and applications to complete these tasks.
• Deep knowledge of data features engineering, data mining, machine learning, or information retrieval.
• Experience processing large amount of structured and unstructured data, including integrating data from multiple sources.
• Experience in production support and troubleshooting.
• You find satisfaction in a job well done and thrive on solving head-scratching problems.

Language requirement: English, Mandarin is plus