Data Engineer

Key Skills

Agile methodologies, Analytical and Problem solving, Ansible, Apache Airflow, Apache Kafka, AWS, Continuous Integration & Continuous Delivery - CI/CD, ETL frameworks, ETL/ELT technologies, Git, Java Programming, Luigi, NoSQL, PostgreSQL, Puppet, Python Programming, Scala Programming, Spark Core, SQL

Job Description

About the Data Team

Athena's data team is a market leader in developing complex ETL and machine learning solutions. With published authors and award winning data scientists who contribute some of the major machine learning and distributed data technologies such as Apache Spark, we are a friendly, passionate group of engineers making a career out of building great software for our customers.

The Adthena data team is dealing with hundreds of millions of data points every day, generated from over two thousand data processes running through workflows, huge distributed computations in spark, and streaming data coming in twenty-four hours a day at hundreds of times a second.

Our engineering culture is underpinned by sharing knowledge, coaching and growing together. You will have the opportunity to explore/innovate new technologies, mentor engineers and lead Technology initiatives. You will enjoy this role if you love writing code, learning cutting edge new technologies, solving problems and winning as a team.

What you'll do

As a Data Engineer, you will be working across our entire stack, so a real passion to drive the product and technology forward is something that we value. Your responsibilities will include helping with a vision for the future architecture of this complex data system and adding innovative ideas that use the latest cutting edge technology. You will work closely with Web and Data Science teams to deliver user-centric solutions to our customers and become an expert in developing high quality technical solutions.

Reporting to: VP of Engineering

Responsibilities

• Understand distributed technologies and the best practices around them
• Build and maintain services/features/libraries that serve as a definitive example for new engineers
• Design and write effective complex Spark jobs (data processes, aggregations, pipeline)
• Design and write complex asynchronous, highly parallel low latency APIs and processes
• Work as part of an Agile team to maintain, improve, and monitor Adthena's data collection processes using Scala and Java
• Apply industry practices such as TDD and SOLID, and Adthena's Engineering Practices
• Understand and be able to apply data structures and algorithms
• Understand Adthena's data architecture and use appropriate design patterns
• Design database(s) and database tables across a range of data storage technologies
• Support our Data Science team to help deliver their machine learning models into production environments
• Be empathetic towards team members and customers

Preferred Qualifications

• Bachelor's degree in Computer Science, similar technical field of study or equivalent practical experience.
• Commercial experience developing Spark Jobs using Scala
• Commercial experience using Java and Scala (Python is nice to have)
• Experience in data processing using traditional and distributed systems (Hadoop, Spark, AWS - S3)
• Experience designing data models and data warehouses.
• Experience in SQL, NoSQL database management systems (PostgreSQL and Cassandra)
• Commercial experience using messaging technologies (RabbitMQ, Kafka)
• Experience using orchestration software (Chef, Puppet, Ansible, Salt)
• Confident with building complex ETL workflows (Luigi, Airflow)
• Good knowledge of working cloud technologies (AWS)
• Good knowledge using monitoring software (ELK stack)
• Motivated problem-solving skills, ability to bring ideas forward and adapt solutions to complex challenges

Technologies we use

• Languages: Java, Scala, JavaScript (React, Backbone), SQL and scripting using Bash and Python
• Frameworks: DropWizard, React, Akka and Play Framework (Scala)
• Databases: PostgreSQL, AWS(S3), Redshift, Redis, MongoDB, Cassandra
• Technologies: RabbitMQ (messaging), Quartz scheduling, Docker and Kubernetes, Maven
• CI/CD: TeamCity, Jenkins
• Source Control: Git (GitHub)
• Other Tools: IntelliJ IDEA, Jira, Grafana

Why join Adthena?

• Work on a multi-award winning product in a position where you are being valued and feel you can make your mark
• Remote-Friendly
• Untracked annual leave
• Flexible working environment
• Receive stock options. Our employees do incredible work, so as we grow everyone shares the rewards.
• Family care package (6 months fully paid maternity & 2 months paternity cover)
• Private Medical Insurance (Vitality Health)
• Home Office Stipend (Up to £200)
• Training & career development pathway
• Monthly Hackathons
• Great culture- helping all employees to learn, grow and develop their skillset
• Join a company with an extremely fast growth trajectory. (We’ve now got three offices globally and we don’t plan to stop there!)
• Free pair of trainers when you join (you choose what ones you want)
• Lots of social activities to join in; Regular off-sites, bake-offs, office table tennis tournaments and much more!
• Strong culture values: Deliver Remarkable Quality, Smart and Always Improving, Freedom with Responsibility, and Get Shit Done!