Software Engineer - Big Data

Key Skills

Ansible, Apache Flink, Apache Hive, Big Data Technology, Debugging, HBase, HDFS, NoSQL, Puppet, Python Programming, Scala Programming, Spark Core, Terraform, YARN

Job Description

We are looking for software engineers to join our agile team responsible for data processing and management for our client data stores and data provided by our real-time bidding solution on petabyte scale. Every day we ingest hundred of data streams from around the globe, process terabytes of data and provide them to our analytics and customer stores building the foundation for our Machine Learning, Analytics solutions within the company, as well as build analytics and insights into the data for our customers.

Essential Responsibilities

• Build fault-tolerant, scalable batch and real-time distributed data processing systems
• Daily use technologies such as YARN, HDFS, Spark, Flink, Kafka, Hive, HBase, OpenTSDB, Vertica, SQL DB
• Orchestrate generic deployments following the “build once run everywhere” approach – on premise or cloud
• Selection and use of adequate cloud technologies to fulfill scalability and performance requirements
• Participate in architecture discussions, influence the road map, take ownership and responsibility over new projects
• Optimize performance and resource utilization on large production clusters
• Maintain and support existing platforms and applications, evolve them to newer tech stacks and architectures
• Contribute to open source projects

Desired Characteristics

• Proven long term experience and enthusiasm for distributed data processing at scale, eagerness to learn new things
• Expertise in designing and architecting distributed low latency and scalable solutions in a hybrid environment – cloud and on-premise
• Exposure to the whole software development lifecycle from inception to production and monitoring
• Fluency in Java or solid experience in Scala, Python
• Expert in usage of services like Spark, Hdfs, Hive, Hbase
• Experience in adequate usage of cloud services (aws) at scale
• Experience in agile software development processes
• Excellent interpersonal and communication skills

Nice To Have

• Experience with large scale / multi-tenant distributed systems
• Experience with columnar / NoSQL databases – Vertica, Snowflake, Hbase, Scylla, Couchbase
• Experience in real team streaming frameworks – Flink, Storm
• Experience with configuration management tools such as Terraform/Puppet, Salt, Ansible
• Experience with debugging and tuning JVM garbage collection and memory problems

About Zeta Global

Zeta Global is a data-powered marketing technology company with a heritage of innovation and industry leadership. Founded in 2007 by entrepreneur David A. Steinberg and John Sculley, former CEO of Apple Inc and Pepsi-Cola, the Company combines the industry’s 3rd largest proprietary data set (2.4B+ identities) with Artificial Intelligence to unlock consumer intent, personalize experiences and help our clients drive business growth.