Data Engineer Specialist

Key Skills

Java Programming, Python Programming, Apache, C++, Node.js, C Programming, SQL, Scrum Agile Methodology, RDBMS, Apache Kafka, Scala Programming, Machine learning techniques, NoSQL, MapReduce, SPARK Programming, RabbitMQ, ETL frameworks

Job Description

SRBR is looking for an experienced Data Engineer to help our team to strive for efficiency while implementing and optimizing data pipeline architectures applied to multidisciplinary research. The ideal candidate has experience with pipeline building in scenarios of high volume and high heterogeneity, and is a data wrangler who enjoys optimizing existing data-centered systems or even building them from scratch.
The Data Engineer will help creating and implementing architectures responsible for extracting/ingesting raw data and producing machine-readable datasets that will feed a myriad of models.

Role and Responsibilities

Responsibilities

• Propose, implement, and maintain optimal data pipeline architectures;
• Design, implement, install, test, and maintain data management and processing systems;
• Collaborate and offer support to software developers, machine learning engineers, and data scientists on troubleshooting and scaling up data-related initiatives, ensuring optimal, consistent solutions;
• Integrate large, heterogeneous, complex datasets in order to meet functional / non-functional domain requirements;
• Efficiently benchmark, select, and combine technological tools, processing systems, databases, algorithms, and programming languages to fit requirements;
• Plan, deploy and maintain data infrastructures;
• Propose and automate data-quality evaluation processes;
• Plan, install/configure, and implement disaster recovery procedures/tools.

Main requirements

• Solid Computer Science, Statistics, Mathematics, Computer Engineering, Electrical Engineering or similar formal education;
• Strong knowledge on developing and deploying data-centered/ETL solutions using Java, Python, NodeJS, Scala, or C/C++;
• Strong knowledge and experience with both software and system architecture design patterns;
• Knowledge and experience with distributed data storage (Hadoop FS) and big data processing engines (Apache Hadoop/MapReduce, Apache Spark, Apache Tez);
• Experience with data ingestion/stream tools (Apache Kafka, RabbitMQ), message protocols (AMQP, MQTT, WebSocket) and ETL tools
• Strong software engineering skills, including methodologies, best practices and related tools;
• Experience applying Agile/Scrum practices and values;
• Experience with data modeling and strong knowledge of SQL/NoSQL DBMSs and query language;
• Proven 3+ years of experience applying required skills in advanced and meaningful set of projects;
• Written and verbal fluency in both Portuguese and English.
   

Skills and Qualifications

Will be considered a plus:

• Experience working on meaningful artificial intelligence projects;
• Experience with data analysis of high volumes of data;
• Experience working with real-time scenarios like IoT, telemetric, etc.
• Experience working on scenarios involving temporal-series data analysis, statistics, and modeling;

Required Education:

• Bs.C. in Computer Science, Statistics, Mathematics, Computer Engineering, Electrical Engineering or other similar titles obtained from top-tier universities.
• Ms.C. or Ph.D. in Computer Science, Statistics, Mathematics, Computer Engineering, Electrical Engineering or similar area obtained from a top-tier university (nice to have).