Lead Data Engineer

Key Skills

Agile methodologies, Algorithms, Apache, Apache Hadoop, AWS, Azure, Big Data Technology, Cloud computing, Data Analysis, Data Modeling, Data structuring, ETL frameworks, Google BigQuery, Google Bigtable, Google Cloud Platform (GCP), JavaScript, Leadership, NoSQL, Postgres, Python Programming, Spark Core, SQL

Job Description

Leads the implementation of information systems to manage financial functions for Patient Financial Services/Accounts. Oversees projects and initiatives to meet user requirements. Serves as a liaison between department (user) and vendors. Ensures systems compliance to applicable laws, regulatory standards and policies.

Job Responsibility

• Design, develop, and maintain data infrastructure on GCP and Azure
• Work with data scientists, analysts, and other cross-functional teams to ensure efficient and reliable data pipelines and systems
• Develop and maintain data pipelines using GCP and Azure services
• Develop and maintain ETL processes to extract, transform, and load data from various sources into BigQuery, Postgres, and Azure SQL Database
• Collaborate with data scientists and analysts to understand their data requirements and provide them with the necessary infrastructure and tools to access and analyze the data
• Optimize and tune data pipelines and ETL processes for performance, scalability, and cost-effectiveness
• Monitor and troubleshoot data pipeline issues, ensuring data availability and reliability
• Implement data security and privacy measures to protect sensitive information
• Stay up-to-date with industry best practices and emerging trends in data engineering, GCP, Azure, and big data technologies
• Document data engineering processes, procedures, and workflows for reference and knowledge sharing
• Coordinates implementation of information system initiatives to meet user requirements. 
• Communicates with user departments and project teams regarding implementation activities and system changes to ensure feasibility. 
• Develops system specifications that appropriately meet department requirements. 
• Develops systems to track data in comprehensive and innovative ways. 
• Collaborates with hospital departments and outside vendors to facilitate managed care contract implementation. 
• Coordinates and maintains communication between the hospital and vendor(s) regarding system applications. 
• Performs periodic post-installation reviews of system initiatives. 
• Conducts system-wide in-services with managers and staff, as necessary. 
• Ensures systems compliance to applicable laws, regulatory standards and policies. 
• Uses knowledge and skills to contribute to the development of functional objectives and principles to achieve goals in innovative ways. Problems addressed are of diverse scope and complexity ranging from moderate to substantial.
• Performs related duties as required. All responsibilities noted here are considered essential functions of the job under the Americans with Disabilities Act. Duties not mentioned here, but considered related are not essential functions.

Job Qualification

• Bachelor’s Degree required, or equivalent combination of education and related experience.
• 5-7 years of relevant experience including technical experience with Apache Hadoop, NoSQL, setting up cloud clusters, Apache Spark, and other advanced data science and big data technologies, required.

HIGHLY PREFERRED

• Experience with GCP and Azure Cloud Environments in terms of designing, building, operationalizing, securing, and monitoring data processing systems.
• Programmatic experience with tools like Python, Apache Spark, Javascript
• Proficient in SQL, Python, and familiarity with BigQuery, Pub/Sub, Dataflow, Bigtable, Dataform, Data fusion, cloud spanner
• Experience designing and developing scalable and efficient data pipelines.
• Strong understanding of database systems, data warehousing, data analysis, and distributed systems
• Deep understanding of data structures, data modeling, and software architecture
• Prior experience working with data pipeline tools.
• Experience working in Agile teams preferred.
• Applied experience and knowledge of Infrastructure as Code required – Terraform experience preferred.
• GCP Cloud Data Engineering Certification
• Experience with FHIR or Healthcare data sets preferred, but not required.

Additional Salary Detail

The salary range and/or hourly rate listed is a good faith determination of potential base compensation that may be offered to a successful applicant for this position at the time of this job advertisement and may be modified in the future.When determining a team member’s base salary and/or rate, several factors may be considered as applicable (e.g., location, specialty, service line, years of relevant experience, education, credentials, negotiated contracts, budget and internal equity).