Senior Data Engineer

Key Skills

AWS, Azure, Confluence, Data Biuld Tool - dbt, Data Warehousing, Docker, ETL/ELT technologies, Github, Google Cloud Platform (GCP), JIRA, Kubernetes-K8s, Monte Carlo Simulation, PostgreSQL, Python Programming, RDBMS, SDLC, Snowflake, SQL

Job Description

With a broader team of more than 250 associates and offices in New York, Chicago, Plano, Silicon Valley, Mexico, India and Poland, we’re unleashing the full potential of automation, data science, and machine learning to challenge the way snacks and beverages are sold every day.We're looking for a senior software engineer to join our PepsiCo eCommerce Catalog engineering team and lead the data engineering in that space. You’ll have a once-in-a-generation opportunity to influence the biggest players in online grocery while also building out data pipelines that are at the core of hundreds of millions of dollars in transactions.

Responsibilities

• You will own development of ETL data pipelines, taking responsibility for designing, data modeling, coding, testing, scalability, operability, and ongoing metrics.
• Collaborate with a globally distributed product/platform team that includes product manager, business stakeholders, UX design, data analysts, data scientists, BI, and data engineers
• You will be the people manager of the Catalog data engineering team
• Assist with hiring the best and brightest engineers to the team
• Help in creating and maintaining a fantastic work environment and culture that attracts and retains top engineering talent

The Catalog engineering team works mostly in Elixir while the data engineering side relies largely on Python: however, the way this evolves is up to you. We put more emphasis on your general engineering skill and willingness to learn than knowledge of a particular language or framework.

Qualifications

• Bachelor’s Degree (Computer Science or related preferred)
• Professional experience using Python or similar programming language
• Professional experience using Elixir or similar programming language
• Strong knowledge of engineering best practices for the full software development life cycle, including coding standards, code reviews, source control management, continuous integrations, testing, and operations
• Deep understanding of RDBMS databases and SQL language
• Understanding of data warehousing
• Exposure to building ETL pipelines
• Knowledge of Airflow, Kubernetes and Docker nice to have
• Hands-on experience with GCP, AWS, or similar nice to have
• Strong verbal and written communication skills

Some technologies that we use. Familiarity is great, but not necessary:

• Python
• Elixir
• Airflow
• Great Expectations
• DBT
• Monte Carlo Data
• Snowflake
• PostgreSQL
• Docker
• Kubernetes (we have a dedicated team for this)

Methodology

• Agile (Lean / Kanban)
• Issue tracking tool: JIRA
• Knowledge repository: Confluence
• Version control system: GitHub
• Communication tool: Slack & Zoom
• Cloud providers: AWS, Snowflake & Azure
• Testing: automated data quality (Great Expectations), performance, unit and DAG testing