Ld Data Reliability Engineer - India

Key Skills

Aartificial intelligence, Apache Hadoop, Apache Kafka, AWS, Azure, Big Data Technology, Continuous Integration & Continuous Delivery - CI/CD, Data science techniques, Git, Google Cloud Platform (GCP), Java Programming, Machine learning techniques, Python Programming, Site Reliability Engineering (SRE), SPARK Programming, SQL

Job Description

Accountabilities:

• Automate data tasks on GCP
• Work with data domain owners, data scientists and other stakeholders to ensure that data is consumed effectively on GCP
• Design, build, secure and maintain data infrastructure, including data pipelines, databases, data warehouses, and data processing platforms on GCP
• Measure and monitor the quality of data on GCP data platforms
• Implement robust monitoring and alerting systems to proactively identify and resolve issues in data systems. Respond to incidents promptly to minimize downtime and data loss.
• Develop automation scripts and tools to streamline data operations and make them scalable to accommodate growing data volumes and user traffic.
• Optimize data systems to ensure efficient data processing, reduce latency, and improve overall system performance.
• Collaborate with data and infrastructure teams to forecast data growth and plan for future capacity requirements.
• Ensure data security and compliance with data protection regulations. Implement best practices for data access controls and encryption.
• Collaborate with data engineers, data scientists, and software engineers to understand data requirements, troubleshoot issues, and support data-driven initiatives.
• Continuously assess and improve data infrastructure and data processes to enhance reliability, efficiency, and performance.
• Maintain clear and up-to-date documentation related to data systems, configurations, and standard operating procedures.

Qualifications:

• Bachelor's or Master's degree in Computer Science, Software Engineering, Data Science or related field, or equivalent practical experience
• Proven experience as a Data Site Reliability Engineer or a similar role, with a strong focus on data infrastructure management
• Proficiency in data technologies, such as relational databases, data warehousing, big data platforms (e.g., Hadoop, Spark), data streaming (e.g., Kafka), and cloud services (e.g., AWS, GCP, Azure).
• Strong programming skills in languages like Python, Java, or Scala, with experience in automation and scripting.
• Experience with containerization and orchestration tools like Docker and Kubernetes is a plus.
• Experience with data governance, data security, and compliance best practices on GCP
• Solid understanding of software development methodologies and best practices, including version control (e.g., Git) and CI/CD pipelines.
• Strong analytical and problem-solving abilities, with a proactive attitude towards identifying and resolving issues.
• Excellent communication and collaboration skills to work effectively in a team-oriented environment.
• Strong background in cloud computing and data-Intensive applications and services, with a focus on Google Cloud Platform
• 10+ years of experience in data engineering or data science on GCP
• Experience with data quality assurance and testing on GCP
• Ability to work independently and as part of a team
• Excellent communication and documentation skills
• Proficiency with GCP data services (BigQuery; Dataflow; Data Fusion; Dataproc; Cloud Composer; Pub/Sub; Google Cloud Storage)
• Strong understanding of logging and monitoring using tools such as Cloud Logging, ELK Stack, AppDynamics, New Relic, Splunk, etc.
• Understanding of current and emerging technologies
• Proven ability to learn new technologies, including open source and cloud-native offerings
• Programming languages e.g., Python, Java, SQL
• Knowledge of AI and ML tools is a plus
• Google Associate Cloud Engineer or Data Engineer certification is a plus
• Excellent oral and written communication skills