Big Data Engineer

Key Skills

Java Programming, Python Programming, SQL, DevOps, Azure, PowerBI, Scala Programming, Tableau, PySpark, SPARK Programming, Artifactory, Data Lake, Synapse

Job Description

Hitachi Solutions is a global Microsoft solutions integrator passionate about developing and delivering industry-focused solutions that support our clients to deliver on their business transformation goals. Our industry focus, expertise, and intellectual property is what truly sets us apart. We have earned, and continue to maintain, a strategic relationship with Microsoft. Recognized for our achievements - teaming with our clients to deliver innovative digital solutions and services - is how we have achieved year after year recognition.

As their trusted advisor, we support our clients to deliver on their strategic business initiatives as they unify, automate, and modernize their data and operations to increase efficiency, reduce costs, and enhance their customer's experience. Our over 3,000 team members across 14 countries, and our 18 years of 100% focus on Microsoft technologies and business applications, is how we deliver excellence through expert services and industry-focused cloud solutions.

A part of Hitachi, Ltd., our company has a long and rich history of innovation, financial strength, and international presence of one of the world's largest companies. Since 1910, Hitachi, Ltd. has been a leader in manufacturing innovative products and solutions that support industry and social infrastructure around the globe supported by 303,000 employees in over 100 countries and across 864 companies.

Job Description

Job Description

DATA SCIENCE AND ARTIFICIAL INTELLIGENCE TEAM

This position in our company is housed in our Data Science and Artificial Intelligence Team. Joining this team represents an opportunity to fast-track your career and to work with a team of fun and nerdy colleagues in a disruptive startup atmosphere: focused on hypergrowth, moving quickly, and making mistakes in the furtherance of innovation and sound engineering.

Armed with an existing book of business, and a stable financial parent - it is the goal of this group to transform our company into a billion-dollar product company, by focusing on engineering excellence and driving customer success.

Big Data Engineer

Hitachi Solutions is seeking a Big Data Engineer to work in our Data Science practice. This is a high-visibility, high-impact, full-time role for professionals with a proven history of operational excellence on data projects, and a leadership position in technology. Individuals applying for this role will be accountable for Hitachi Solutions' client-side rollouts, solution design, and delivery of engineering requirements in support of the team.

Your Responsibilities

As a Big Data Engineer, you will work closely with Data Scientists to deploy machine learning models and troubleshoot models in production environments. You will be responsible for designing and building solutions using various Azure technologies to help clients scale their data analytics and machine learning capabilities. In this role, you will:

• Execute. This means being constantly biased towards action, knowing the data, being in the details, setting a high bar and building strong collaborative relationships with other teams, thinking boldly, disambiguating, always thinking of the customer first, and rapidly iterating towards something demonstrably awesome.
• Designs and implements Build and Release pipelines for Continuous Integration/Continuous Deployment (CI/CD) in Azure DevOps for various Microsoft Azure services.
• Lead implementation from ideation to deployment of core data management capabilities
• Build and maintain hard-working relationships with data vendors & core internal development partners to achieve a comprehensive, quality data offering
• Instill excellence into the processes, methodologies, standards, and technology choices embraced by the team
• Identify and articulate technical and operational requirements for new feature developments and platform investments
• Drive design and planning activities by removing ambiguities, resolving dependency questions, and providing high level engineering estimates
• Design and implement engineering pipelines to support various Data Science and Artificial Intelligence solutions

Qualifications

Preferred Qualifications

• Strong interpersonal and communication skills to translate technical concepts for business groups, present to executive audiences, and drive effective discussion within a team
• Ability to collaborate effectively in cross-functional teams and work independently
• Ability to prioritize and thrive in a fast-paced, dynamic environment
• Hands-on experience with programming in Spark, Python, pySpark, Scala, or SQL
• A Bachelor's Degree in computer science, information systems, engineering related fields or equivalent professional experience
• Experience designing, building, and scaling streaming and batch data pipelines
• Experience working with Databricks Lakehouse architecture and principles
• Experience with Azure cloud technologies such as Azure Databricks, Azure Data Factory, Azure DevOps, Lakehouse, Azure Synapse, Azure Data Lake and similar tools on the Azure stack
• Experience with leading high performing data engineering teams with a history of on-time delivery, especially in operations and strategic role
• Expertise in Agile development methodologies and their adoption within teams
• Experience with data visualization and dashboarding tools, such as Tableau or Power BI
• Data modeling and Kimball methodology