Data Platform Engineer

Key Skills

Big Data Technology, Data Engineering, Data science techniques, Design, ETL/ELT technologies, GoLang, Java Programming, Python Programming, Scala Programming, SPARK Programming, SQL, Streaming APIs

Job Description

The Compute and Workflow Infrastructure team is our data platform team responsible for providing frameworks and services for operating on our data including the use of Spark, Trino, and Druid among others. The team also develops and manages systems that schedule and execute thousands of container based workflows on containers in the Elastic Kubernetes Service (EKS) provided by Amazon Web Services (AWS).

• Our infrastructure is 100% deployed on AWS
• We make heavy use of Spark to process and transform data
• We use several Trino clusters for ad-hoc queries and analysis
• We also use Druid clusters for specific workloads, as well as some custom developed services for working with data
• We run 1000s of batch jobs each night, training 100s of models that feed our recommendation engines and other data driven APIs
• The source of truth for our data warehouse is AWS S3, and we use the Hive Metastore to manage schemas, and power other systems for data exploration and discovery
• We write most of our code in JVM languages (Java and Scala), Python, and golang

ABOUT THE ROLE

In this role, you will be developing, monitoring and improving our services, libraries and tools that operate on our data. In addition, you’ll consult with Data Scientists and Analysts to build robust pipelines that take advantage of Spark and our data ecosystem.

• This is an individual contributor role on the compute and workflow infrastructure team, part of the larger data platform team in our tech organization
• You will help enhance our Spark infrastructure and help develop libraries that improve the reading and writing of data and enhance user capabilities with functionality like User Defined Functions (UDFs).
• You’ll create services and tools to help make the experience better for our data scientists, and create the abstractions they need to effectively use our data ecosystem
• You’ll help us improve our Spark and Trino deployments to function well under load and in AWS.
• You will help make our infrastructure more scalable, more reliable, and easier to use
• You’ll consult with others on the team, helping them with some of their daily data challenges

YOU’RE EXCITED ABOUT THIS OPPORTUNITY BECAUSE YOU WILL…

• You’ll have opportunities to work on high impact projects that improve data availability and quality, and provide reliable access to data for analytics, machine learning, and the rest of the business
• You’ll build services to ingest data into our warehouse and ensure it’s clean and consistent.
• You’ll provide ETL patterns that others can follow, abstractions to make working with data easier, and consult with other team members to create new data pipelines and improve them.
• Many of the changes we need would also benefit others in the big data community. You’ll have the opportunity to contribute back. 

WE’RE EXCITED ABOUT YOU BECAUSE YOU…

• You have 5 or more years of relevant software experience with significant contributions.
• You have exceptional coding and design skills, particularly in Java/Scala and Python or Golang.
• You’ve used Spark extensively and are comfortable with the Hive Metastore. You know how to take advantage of Spark APIs as well as SQL.
• You’ve worked on some challenging data migration and data transformation projects.
• You work autonomously and take ownership of projects.
• You understand how big data infrastructure works in the public cloud.
• You are naturally curious and get excited to dig in and understand how things work.

WHY YOU'LL LOVE WORKING AT STITCH FIX...

• We are a group of bright, kind people who are motivated by challenge. We value integrity, innovation and trust. You’ll bring these characteristics to life in everything you do at Stitch Fix.
• We cultivate a community of diverse perspectives— all voices are heard and valued.
• We are an innovative company and leverage our strengths in fashion and tech to disrupt the future of retail. 
• We win as a team, commit to our work, and celebrate grit together because we value strong relationships.
• We boldly create the future while keeping equity and sustainability at the center of all that we do. 
• We are the owners of our work and are energized by solving problems through a growth mindset lens. We think broadly and creatively through every situation to create meaningful impact.
• We offer comprehensive compensation packages and inclusive health and wellness benefits.

COMPENSATION AND BENEFITS

Our anticipated compensation reflects the cost of labor across several US geographic markets, and the range below indicates the low end of the lowest-compensated market to the high end of the highest-compensated market. This position is eligible for new hire and ongoing grants of restricted stock units depending on employee and company performance. In addition, the position is eligible for medical, dental, vision, and other benefits. Applicants should apply via our internal or external careers site.

Salary Range

$186,000—$199,000 USD