Senior Systems Software Engineer - Performance - Accelerated Apache Spark

Key Skills

Apache Hive, Apache Kafka, Big Data Technology, CUDA/GPU programming, Data science techniques, Effective communication skills, GPUs, Java Programming, Machine learning techniques, Presto, Python Programming, PyTorch, Scala Programming, SPARK Programming, SQL, TensorFlow, Writing

Job Description

We are seeking experienced System Software Engineers adept at Apache Spark to join our team to help drive innovation for accelerated data processing on GPUs. As Nvidia is leading the world in accelerated computing, we are building the next generation data processing ecosystem. Data scientists spend a considerable amount of time exploring data and iterating over machine learning (ML) experiments. Every hour of compute required to sort through datasets, extract features and fit ML algorithms impedes an efficient business workflow. Nvidia believes that data science workflows can benefit tremendously from being accelerated, to enable data scientists to explore many more and larger datasets to drive towards their business goals, faster and more efficiently.

At NVIDIA, we are passionate about working on hard problems that have an impact. You will need to have previous experience working with Apache Spark applications, implementing big data applications for a variety of customers, programming skills, and familiarity with open source big data processing frameworks. You will work with an engineering team accelerating Apache Spark with GPUs using CUDA and open source libraries. This is a strategic investment for Nvidia. The code is being adopted by multiple cloud service providers and Apache Spark distributions. As Nvidia leads the way in providing the data center of the future with a CPU-GPU superchip platform, accelerating data processing will be a need for all enterprises.

What you'll be doing:

• Scale Apache Spark acceleration for high-end GPUs and with the new Grace Hopper superchip platform

• Work with select customers and partners to make Apache Spark applications accelerated with GPUs across on-prem and cloud environments

• Benchmark and optimize Apache Spark workloads on GPUs for accelerated performance

• Build tools to analyze data processing workloads to identify opportunities for acceleration and cost savings

• Help define a reference architecture for next-generation computing cloud

• Work with major cloud service providers and Spark vendors globally to enable accelerated computing for data processing use cases

What we need to see:

• 8+ years of of experience in software development, with the majority in data processing

• 3+ years hands on experience with Apache Spark application design and development

• BS/MS/PhD in computer science or a related field

• Familiarity with the modern data open source ecosystem (Apache Hadoop, Flink, Presto, Hive, Kafka, TensorFlow, PyTorch etc)

• Deep understanding of performance optimization in low-level environments

• Experience in writing performant distributed software at scale

• Experience working on public and private cloud platforms

• Prior experience supporting enterprise customers

• Knowledge of SQL, Python and Scala/Java

• Excellence at communicating, presenting and explaining technical topics

Ways to stand out from the crowd:

• Familiarity with RAPIDS and UCX

• Experience with performance optimizations in CSPs for big data use cases

• Understanding and expertise in hardware configuration options as it pertains to performance and cost

• Experience with working with and developing on GPUs

We are widely considered to be one of the technology world’s most desirable employers, and as a result have some of the most forward-thinking and hardworking people in the world working for us. If you're passionate, creative, and driven, we'd love to have you join the team. With competitive salaries and a generous benefits package, we are widely considered to be one of the technology world’s most desirable employers. We have some of the most forward-thinking and hardworking people in the world working for us and, due to unprecedented growth, our exclusive engineering teams are rapidly growing. If you're a creative and autonomous engineer with a real passion for technology, we want to hear from you.

The base salary range is 216,000 USD - 333,500 USD. Your base salary will be determined based on your location, experience, and the pay of employees in similar positions.

You will also be eligible for equity and benefits. NVIDIA accepts applications on an ongoing basis.

NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.