Data Engineers

Key Skills

Apache Kafka, Dash framework, Database, Design, Java Programming, Microsoft-Excel, MySQL, NoSQL, Python Programming, SPARK Programming, SQL, SQL Server, Vertica

Job Description

What to Expect

The Data Engineer is responsible for processing structured and unstructured data, validating data quality, and developing and supporting data products. We are looking for someone with strong hands on experience in all layers of data Integration and analytics! We especially need experience in using Python as an ETL tool. The Data Engineer plays a significant role in Agile planning, providing advice and guidance, and monitoring emerging technologies. Some of the technology we use: Python, Informatica, SQL Server and MySQL, Vertica, Kafka

What You’ll Do

• Design, code, test, correct and document programs and scripts using agreed standards and tools to achieve a well-engineered result
• Derive an overall strategy of data management, within an established information architecture that supports the development and secure operation of existing and new information and digital services
• Plan effective data storage, security, sharing and publishing within the organization
• Gathers and processes raw, structured, semi-structured, and unstructured data using batch and real-time data processing frameworks
• Ensures data quality and implements tools and frameworks for automating the identification of data quality issues
• Collaborate with internal and external data providers on data validation providing feedback and making customized changes to data feeds and data mappings
• Mentor and lead data engineers providing technical guidance and oversight
• Provides ongoing support, monitoring, and maintenance of deployed products

What You’ll Bring

• Strong experience with relational databases like SQL Server, MySQL and Vertica. NoSQL databases experience is a plus!
• Strong background in data modeling, data access, and data storage techniques
• Experience with design, development, and implementation of highly scalable, high-volume software systems and components, source of truth systems for different business areas, developing and maintaining web services in an agile environment
• Working experience with Kafka Streaming layer
• Experience in Spark Framework on both batch and real-time data processing is a plus
• Experience in Big Data Integration & Analytics is a plus
• Experience in Supply Chain and Logistics data is a plus