Data Engineer

Key Skills

Java Programming, Design, Python Programming, Database, SQL, Apache Kafka, A/B Testing, NoSQL, MapReduce, Google Cloud Platform (GCP), Apache Impala, Amazon RedShift, Storm, DynamoDB, HBase, Streaming APIs, Data pipelines, Large data sets

Job Description

• Design, build and own the core data models and systems that able to support petabyte of data pipeline across the Tencent's overseas games.
• Analyze large amounts of players’ data with business use cases to bridge the gaps of data-driven lifecycles.
• Partner with multiple internal teams and external partners to collect requirements and deliver large-scale data solutions for various analytics and business applications.
• Collaborate with product managers to define, create, and maintain data applications and platform such as real-time platforms, data pipeline, telemetry implementation, and machine learning deployment to help achieve Tencent's publishing goals.
• The challenges range from building real-time and offline distributed data processing pipelines for feature engineering and automation as well as building systems and frameworks to deploy, scale, test, manage and monitor real-time machine learning models and user acquisition algorithms.

Qualifications:

• Advanced degree in Computer Science, Engineering, Mathematics or equivalent technical field.
• Programming and scripting language (e.g. SQL, Python, Java/Scala, Golang).
• 4+ years of professional software development: experience in backend development experience with interest in work involving data pipelines, distributed systems, performance analysis, and large-scale data processing.
• Experience working with distributed data technologies (e.g. Hadoop, MapReduce, Spark, Flink, Kafka, Impala, HBase etc.) for building efficient and large-scale data pipelines.
• Experience shipping scalable data solutions in the cloud (AWS, Azure, GCP) and database technologies such as Snowflake, Redshift, SQL/NoSQL and or columnar databases.
• Dynamic team player with ability to manage numerous requests concurrently and strategically, prioritizing when necessary.
• Strong problem-solving skills with an ability to isolate, deconstruct and resolve complex data engineering challenges.
• Ability to communicate with worldwide business stakeholders regard data products and technical requests.
• Proven track record to clearly form and communicate ideas to both technical and non-technical worldwide audiences.

Bonus:

• Experience strongly preferred building stream-processing applications using Apache Flink, Spark-Streaming, Apache Storm, Kafka Streams or others.
• Explore modern data storage, messaging, and processing tools (Kafka, Spark, Hadoop, Cassandra, etc.) and demonstrated experience designing and coding in big-data components such as DynamoDB or similar.
• Familiar with concepts related to testing and maintaining models in production such as A/B testing, retraining, monitoring model performance.