Data Platform Engineer

Key Skills

Amazon RedShift, Apache Airflow, Apache Kafka, AWS, Azure, Docker, DynamoDB, Elasticsearch, ETL/ELT technologies, Google Cloud Platform (GCP), MongoDB, MySQL, NoSQL, PostgreSQL, Python Programming, RDBMS, Snowflake, SPARK Programming, SQL

Job Description

Job responsibilities:

Building, maintaining, and monitoring sustainable infrastructure with a focus on developing and maintaining high-volume, low-latency data platforms using latest opensource frameworks, highly available RESTful services, and back-end Cloud based/open source systems

• Execute day to day operation of the data platform to ensure top-notch performance of data platform systems for customer-facing applications
• Work on features to make the data Platform more robust and resilient
• Drive automation of application deployment for production environments
• Identify and support areas for process and efficiency improvement within platform operations
• Taking ownership of your projects, and have the work in small cross functional teams to drive projects to closure
• Highly collaborative with a strong belief in continuous improvement.
• Ability to stay focused and calm in a fast-paced environment.

Requirements to be successful in this role:

• BE/B-Tech/BS/Meng/MSc in computer science or a related technical field from top-tier colleges or equivalent practical experience
• 2-3+ years of experience working on platform infrastructure based on Cloud and open-source systems and automation.
• Must be very good with at least 1 programming lang), preferred: Python
• Django/Flask (preferred, must be experienced in building APIs)
• Databases: SQL querying & db understanding (RDBMS: mysql, postgresql; Nosql: mongodb, elasticsearch, dynamodb)
• Familiarity with big data tools & distributed systems (spark, redshift, kafka)
• Cloud Computing (Any of AWS/Azure/GCP), preferred: AWS
• Orchestration: airflow, docker

Good to Have:

• Monitoring/Logging frameworks (ELK, Grafana)
• Scalable Compute solutions (Kafka, Snowflake)
• Workflow management libraries and tools
• Container orchestration systems like Kubernetes
• Code-as-Infrastructure/Pipeline-as-Code (Terraform/Chef)
• Understanding and functional experience in Big Data Engineering practices: Data
• Warehousing, ingestion pipelines, ELT, data migration, QC

₹14,00,000 - ₹20,00,000 a year