Backline Technical Solutions Engineer (Spark)

Key Skills

Java Programming, Python Programming, Azure, AWS, Data science techniques, Google Cloud Platform (GCP), AI technology

Job Description

As a Backline Technical Solutions Engineer you will enable our customers to be successful with the Databricks platform by resolving critical technical escalations that come from customers as well as the frontline support team. You will be the technical bridge between the frontline support team and the engineering team, thereby becoming the first line of defense for engineering. You will ensure that all issues are vetted by you before it reaches the engineering team. Along with this you will also ensure that the global support team stays updated via deep dive technical trainings, workshops and other enablement programs.

Outcomes

• Troubleshoot, resolve and if required suggest deep code-level analysis of Spark to address complex customer issues related to spark core internals, spark sql, structured streaming, databricks delta and other data engineering and product areas
• Provide best practices guidance as well as suggest appropriate programming usage of Spark core libraries and APIs for custom-built solutions developed by Databricks customers
• Build detailed troubleshooting guides and runbooks to enhance the efficiency of the overall support team.
• Contribute to automation and tooling initiatives to make day to day troubleshooting effective, consistent and efficient.
• Liaise with the Spark Engineering Team within Databricks and ensure that the support team is aware of upcoming features and releases. 
• Proactively reach out to the front line team members and provide technical assistance to long running customer cases.
• Deliver proactive suggestions for improving Spark runtime performance and efficiency in customer-specific environments
• Pinpoint specific areas of the Spark code base that are being affected by known bugs and try to suggest possible workarounds.
• Strengthen your understanding of the entire Spark code base and work towards becoming an Apache Spark contributor/committer.
• Demonstrate true sense of ownership and coordinate with engineering and escalation teams to achieve timely resolution of customer issues and requests
• Be a team player and coach other team members as needed
• Participate in weekend and weekday on call rotation

Competencies

• Minimum 6 years’ experience designing, developing, testing, and sustaining Python or Java or Scala based applications required. Expert level knowledge in python/scala is desired.

• High level understanding of the Apache Spark code base is desired. 

• Must be comfortable with compiling and building the Apache Spark source code.

• Ability to identify and apply patches/bug fixes to the Apache Spark source code is desired.

• Hands on experience in Big Data/Hadoop/Spark/Kafka/Elasticsearch data pipelines required.

• Solid hands on experience with SQL-based database systems is preferred.

• Strong Linux/Unix administration skills are required.

• Strong skills in Linux JVM, GC, Thread dump based troubleshooting is required.

• Hands-on experience with AWS or Azure related services is preferred.
• Candidates must possess excellent written and oral communication skills.
• Good technical skills in Programming, data structures and algorithms is preferred.
• Demonstrated analytical and problem-solving skills, particularly those that apply to a “Distributed Big Data Computing” environment.
• Bachelor’s degree in Computer Science or a related field is required.