Software Engineer (ML), Lucene Search Engine Team, Amazon

Key Skills

Apache Hadoop, Apache Hive, Apache Pig, AWS, Big Data Technology, Data Mining, Machine learning techniques, Natural Language Processing (NLP), Search Engine, Software Development, Spark Core, Statistics

Job Description

Our search engine is designed to support both lexical and semantic matching using state-of-the-art (SOTA) approximate near neighbor (ANN) search on learned vector representations (embeddings).

As a member of this team you will get a chance to dive deep into search engine internals and improve its lexical and semantic matching capabilities using latest advancements in Large Language Models (LLMs) and Vector Search.

You will also get a chance to partner with teams building conversational search experience on top of our Search Engine.

Key job responsibilities

• Improve the semantic matching capabilities of Search Engine using latest advancements in Large Language Models (LLMs) and Vector search
• Dive deep into search engine internals to identify opportunities for improving performance and efficiency of Search Engine.
• Design and develop tooling to improve operational efficiency and participate in oncall rotation.
• Contribute to open source Lucene and bring latest advancements in Lucene in house.
• Collaborate with partner teams to turn feature requests into requirements and work with Senior and Principal engineers to define project roadmap.
• Own the end-to-end delivery on medium complexity projects.

About the team

The Lucene Search Engine team designs and develops query serving and indexing components of Amazon product search stack. Our mission-critical service is built with open-source Lucene at its heart.

The service is responsible for matching customer keywords to products in Amazon’s catalog, apply business rules to narrow down the selection and rank selected products using sophisticated algorithms all under a strict latency budget.

We are open to hiring candidates to work out of one of the following locations:

Palo Alto, CA, USA

BASIC QUALIFICATIONS

• 3+ years of non-internship professional software development experience
• 2+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience
• Experience programming with at least one software programming language
• Experience in machine learning, data mining, information retrieval, statistics or natural language processing
• 2+ years of big data technologies such as AWS, Hadoop, Spark, Pig, Hive, Lucene/SOLR or Storm/Samza experience

PREFERRED QUALIFICATIONS

• 3+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
• Bachelor's degree in computer science or equivalent
• 2+ years of building large-scale machine-learning infrastructure for online recommendation, ads ranking, personalization or search experience

Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.

Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $115,000/year in our lowest geographic market up to $223,600/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, please visit https://www.aboutamazon.com/workplace/employee-benefits. Applicants should apply via our internal or external career site.