Data Engineer

Key Skills

Amazon Athena, Amazon RedShift, Amazon Simple Storage Service (S3), Apache Hadoop, AWS Aurora, Azure, Data Fabric, Databricks, ETL/ELT technologies, Google BigQuery, Google Dataflow, Hyperspectral , PySpark, Python Programming, SQL, Synapse

Job Description

Job Description:

• Designs and develops Data and Analytics solutions
• Performs data analysis, data architecture, data engineering activities
• Designs and builds data ingestion pipelines; develops data transformation, cleansing and preparation processes
• Designs and builds data storage solutions including database objects and data structures
• Takes ownership of complex data-level or database incidents and problems and provides resolutions in a timely manner
• Provides advisory and consultation services to determine the right solution
• Utilizes best practices, work plans, checklists and defined processes
• Proactively supports application teams
• Performs optimization
• Participates in project technical reviews
• Leads or participates in work product reviews
• Engages SMEs or vendors to provide assistance to resolve more complex issues as required

Technology and Functional Skills:

• Azure data platform and components: Azure Data Factory, Databricks, Blob Storage, Synapse, SQL DB/DW, Event Hubs, etc
• AWS data platform and components: S3, Redshift, Spectrum, Athena, Aurora, CLI, etc
• GCP data platform and components: BigQuery, Dataflow, Dataprep, Pub/Sub, etc
• Big Data: Hadoop, Hadoop ecosystem, Spark, Scala, etc.
• Language and Toolset: SQL, Python, PySpark, ELK (Elasticsearch, Logstash, Kibana)
• Data Integration, Extract-Transform-Load (ETL), Data Ingestions, Data Cleansing, Data Preparation, Database
• Data Warehouse, Data Lake, and Business Intelligence (BI) concepts