Cloud Data Engineer

IBM
Apply Now

Job Description

Your Role and Responsibilities

  • Highly skilled Big Data professional with relevant experience developing distributed applications within the Hadoop ecosystem.
  • Adept at implementing Spark using Python and SparkSQL for efficient data processing and testing.
  • Demonstrated expertise in Azure Data Bricks and Azure Data Lake, leveraging these platforms to perform advanced data analytics tasks.
  • Proficient in utilizing various big data processing technologies like HDFS, Hive, Sqoop, and Spark to handle and analyze large-scale datasets.
  • Experienced in integrating Hive queries into Spark environments through Spark SQL, ensuring seamless data operations.
  • It’s a client facing role and hence the consultant should have good communication and interpersonal skills.

Required Technical and Professional Expertise

Big Data professional with over 5 + years of experience developing distributed applications within the Hadoop ecosystem with Cloud (Azure) expertise.

  • Hadoop Ecosystem: HDFS, Hive, Sqoop, Spark 3.0, HDP 2.6.x
  • Big Data Processing: Python, SparkSQL, Spark Streaming
  • Data Formats: AVRO, JSON, ORC, PARQUET
  • Cloud Platforms: Microsoft Azure, ADLS Gen1/Gen2, Azure Data Bricks
  • Data Ingestion / ETL: NiFi, Sqoop, Azure Data Factory (ADF)
  • Azure Cognitive Services: Speech Recognition, Language Understanding
  • Operating Systems: Windows Server, Windows, Mac OS, Linux

Preferred Technical and Professional Expertise

IDE: Eclipse, PyCharm, Visual Studio

AI and NLP: OpenAI GPT Models, Text Generation, Language Translation, Chatbots.

Company Info.

IBM

IBM is a leading cloud platform and cognitive solutions company. Restlessly reinventing since 1911, we are the largest technology and consulting employer in the world, with more than 290,000 employees serving clients in 177 countries. IBM Research provides unparalleled insight into business, industry and society by leveraging advanced computing architectures and methodologies to solve some of the world’s most pressing challenges.

  • Industry
    Information Technology,Computer software,Computer hardware
  • No. of Employees
    292,500
  • Location
    New Orchard Road, Armonk, New York, NY 10504, USA
  • Website
  • Jobs Posted

Get Similar Jobs In Your Inbox

IBM is currently hiring Cloud Data Engineer Jobs in Dallas, TX, USA with average base salary of $78,000 - $145,000 / Year.

Similar Jobs View More