Data Engineer

Key Skills

Java Programming, Oracle, Python Programming, MongoDB, Database, SQL, API, Scala Programming, Apache Cassandra, ETL/ELT technologies, NoSQL, Apache Airflow, Docker, Kubernetes-K8s, Continuous Integration & Continuous Delivery - CI/CD, R Programming, SPARK Programming, DataOps, Data pipelines, Julia

Job Description

The Predictive Analytics and Modeling team is part of the Health Plan group within Kaiser Permanente (voted the #1 Healthcare provider by LinkedIn). Our mission is to develop business recommendation apps using AI/Machine learning to drive data-driven decisioning at Kaiser Permanente. We are hiring a Data Engineer, who will have the unique opportunity to develop data pipelines that power our predictive analytics models for sales recommendation apps that attract customers and drive membership growth while providing high-quality affordable healthcare. The diversity of data allows you to apply cutting-edge processes and technologies to solve problems using machine learning, time series, and consumer behavior analysis.

Essential Responsibilities - What You'll Do:

• You will be the cornerstone of a robust data stack, integrating diverse sources (structured/unstructured) and serving up data that power AI/Machine Learning solutions.
• Build, develop, implement and execute extensible reusable data pipelines consisting of multiple acquisition sources and integration into use case-driven endpoints
• Lead design activities in partnership with data scientists, analysts, and product owners to translate functional requirements into technical specifications for scalable data pipelines
• Oversee management of analytical data assets for exploratory and early-stage analytic usage patterns, and develop recommendations to integrate with production pipelines
• Maintain and optimize workloads in various deployment stages and data environments to ensure optimal performance as data volume and variety increase
• Orchestrate data pipelines using modern tools and techniques to automate repeatable ETL processes, minimize error-prone dependencies and improve the integrity of published data assets
• Collaborate with internal IT teams to troubleshoot incidents and coordinate resolutions to minimize disruption of analytic applications
• Monitor data consumption patterns and develop enhancements to ensure pipelines adapt to evolving data schema and analytic use cases
• Pursues self-development and effective relationships with others by sharing resources, information, and knowledge with coworkers and customers; listening, responding to, and seeking performance feedback; acknowledging strengths and weaknesses; assessing and responding to the needs of others; and adapting to and learning from change, difficulties, and feedback.

Completes work assignments by applying up-to-date knowledge in subject area to meet deadlines; following procedures and policies and applying data, and resources to support projects or initiatives; collaborating with others, often cross-functionally, to solve business problems; supporting the completion of priorities, deadlines, and expectations; communicating progress and information; identifying and recommending ways to address improvement opportunities when possible; and escalating issues or risks as appropriate.

Minimum Qualifications:

• Bachelors degree in Mathematics, Statistics, Engineering, Social/Physical/Life Science, Business, or related field OR Minimum two (2) years experience in data analytics or a directly related field.

• N/A

Preferred Qualifications:

• A college degree in computer science, engineering, data management, information systems or related quantitative field
• 2+ years of experience in data ingest / acquisition and engineering disciplines. Industry experience in health care sector preferred.
• 2+ years of experience and high proficiency with relational databases (Oracle, MS SQL), NoSQL databases (MongoDB, Cassandra), and distributed computing platforms
• 1 year experience and High proficiency working with large, heterogeneous datasets in building/optimizing data pipelines using ETL / ELT, data replication, API access, data virtualization, stream data integration, and emerging technologies.
• 2 year experience and Proficiency in Python, R, Scala, Julia or equivalent scripting language for data analysis
• High proficiency with CI/CD tools and rigorous application of DataOps principles
• Successful track record implementing complex automated data pipelines with commercial data preparation tools (Trifacta) or open-source technologies (Airflow, Spark)
• Demonstrated ability to work across multiple deployment environments, operating systems and through containerization techniques such as Docker and Kubernetes
• Ability to partner, collaborate with, and influence relevant stakeholders across diverse functions and experience levels
• Strong independent judgment, critical thinking and problem-solving skills required to