Senior Cloud Data Engineer

Key Skills

Java Programming, Python Programming, Database, SQL, DevOps, Azure, AWS, RDBMS, Jenkins, Business Intelligence, NoSQL, Apache Airflow, Google Cloud Platform (GCP), Amazon Athena, Elasticsearch, DataOps, Databricks, AWS Kinesis, Data Engineering, Data Lake, Data pipelines, Time series, GitLab, AWS Glue

Job Description

SAIC is seeking a Senior Cloud Data Engineer to work remotely anywhere within the U.S.

Responsibilities:

• Leads key data engineering initiatives, including end to end designing and deploying cloud native data products
• Strong qualifications with Data Management, Data Engineering, Data Architecture, Data Governance, and Data Science Effectively communicate between developers, architects, managers, and end users
• Create a repository/library of data engineering capabilities
• Experiment with cloud native, open source tools and advise on new tools in order to determine optimal solution given the requirements dictated by the use case
• Develop and implement patterns to leverage cloud-computing resources to deploy/ optimize data engineering solutions on digital assets
• Develop and implement DevOps and DataOps pipelines in AWS, Azure, and GCP to enable near-real time delivery of data analytics to end consumers
• Use advanced programming skills in scala, python or any of the major languages to build software solutions/services, including robust data pipelines and dynamic systems
• Write complex etl (extract / transform / load) processes, preferably in python, designs database systems and develops tools for real-time and offline analytic processing
• Develop frameworks, standards & reference material for design and associated products
• Act as a mentor to junior team members to provide technical advice, apply systems and products to consult and advise on additional efforts across multiple domains spanning broader product development
• Collaborate with cloud engineering and data science teams to transform data and integrate algorithms and models into highly available, production systems
• Use in-depth knowledge on hadoop architecture, spark, hdfs hive, pyspark and experience designing & optimizing queries to build scalable, modular, and efficient data pipelines.

Qualifications

Qualifications:

• Must be a US Citizen or lawful permanent resident
• Bachelor's degree in engineering and 14 years or more in related experience. Master's degree in engineering and 12 years or more in related experience. In lieu of a degree, equivalent experience will be considered
• Five (5) or more years of experience as Computer Programmer, Application Developer
• Minimum of four (4) years of experience related to DevOps and/or DataOps Data Engineering, Data Management, Analytics, and/ or Machine Learning certificates
• At least four (4) years’ experience working with cloud based solutions, various open source based frameworks, analytic tools and recognizing their weaknesses

Required Skills:

• Strong experience developing DataOps pipelines using any one of the following:
• Azure using Azure Data Factory and Databricks, EventHub
• AWS using EMR, Glue/Data Pipelines, Amazon Athena, Kinesis
• Deep knowledge and experience utilizing Spark data frames, manipulating hdfs data using python and pyspark. manipulating relational data using spark sql, performance fine turning and handling of large datasets
• Experience with a variety of Database technologies: RDBMS, NoSQL, Search, Data Lakes, Time Series, etc.
• Hands-on experience working on any one of the cloud environments and Automation
• AWS, Azure, GCP
• Hands on Automation tools experience using any one of the following:
• Azure DevOps, Jenkins, Gitlab CI/CD)
• Must also have experience working with any one of the following:
• Bash shell scripts and ARM templates, Amazon Cloud Formation, Terraform
• ETL/ELT or data warehousing experience working on the design, development and documentation of large-scale data objects from disparate data sources

Desired Skills and Certifications:

• Experience using Apache/NiFi, Apache Accumulo, Apache AirFlow
• AWS/GCP/Azure Data Engineer Certification
• Strong experience utilizing streaming data --specifically using Kafka
• Container Management Platforms
• Two (2) years of experience operating in an agile environment