Sr. Databricks Developer

Key Skills

Apache Hadoop, AWS, Big Data Technology, Data Engineering, Data Lake, Data Modeling, Databricks, Design, Effective communication skills, Employee life cycle, ETL/ELT technologies, IT, NoSQL, PySpark, Python Programming, Scrum Agile Methodology, SPARK Programming, Spark-SQL, SPARQL, SQL, SQL Server

Job Description

Sr. Databricks developer responsible for designing and implementing very large scale data engineering solutions in Databricks. You will be responsible to review business requests and translate them into technical solution and technicial specification. In addtion you will work with team members to mentor fellow developers to grow their knowledge and expertize. You will be working in a fast paced and high-volume processing environment, where quality and attention to detail are vital.

Responsibilities

• Design and implement data engineering solutions in big data space 
• Design and implement batch and streaming solutions to support near real-time analytics
• Collaborate with architects, tech leads, QA, business stakeholders, and project managers to design and develop solutions in AWS Databricks.
• Develop end-to-end Enterprise grade solutions, facilitate discussions with business & IT stakeholders and highlighting the outcomes that can be delivered through these solutions.
• Develop data models based on the source systems
• Part of 24x7 on-call roration process for production support
• Troubleshoot productions issues and bug fixes
• Translate business requirements into technical specs
• Hands-on developer as part of a global team and coach fellow developers

Qualifications

• 8+ years of IT experience in big data space like Hadoop, data lake, data engineering using Python & Spark programming languages
• Minimum 3 year of experience in Databricks workspace, Databricks notebooks, Job cluster, Delta Lake, Databricks Lakehouse and Unity catalog
• Minimum 5 years' experience in PySpark and Python development
• 6+ year's design and/or implementation of data pipelines
• Experience on design and develop Data pipelines and ETL or ELT jobs to ingest and process data in data lake
• Strong working knowledge of SQL Server, No SQL, Spark SQL, Data Modeling, Identity & Access Management, Query optimization, Parallel processing
• Must have experience in problem-solving, verbal and communication skills.
• Experience in processing streaming data using Kafka / Pub-Sub
• Experience in AGILE development, SCRUM and Application Lifecycle Management (ALM)
• Must have knowledge in Datawarehouse concepts
• Hands-on experience in full life-cycle of software development or methodology using Agile Scrum/Kanban etc; requirements analysis, design, development testing and implementation
• Plus to have experience in IaaS using Terraform
• Plus to have experience in pandas and tensor flow
• Plus to have Java, JavaSparak experience.
• Databricks certifications is a plus

Labcorp is proud to be an Equal Opportunity Employer:

As an EOE/AA employer, Labcorp strives for diversity and inclusion in the workforce and does not tolerate harassment or discrimination of any kind. We make employment decisions based on the needs of our business and the qualifications of the individual and do not discriminate based upon race, religion, color, national origin, gender (including pregnancy or other medical conditions/needs), family or parental status, marital, civil union or domestic partnership status, sexual orientation, gender identity, gender expression, personal appearance, age, veteran status, disability, genetic information, or any other legally protected characteristic. We encourage all to apply.