Senior Big Data Software Engineer

Key Skills

Python Programming, Scala Programming, C++, Java Programming, Machine learning techniques, Data science techniques, SQL, Apache Hadoop, MapReduce, TensorFlow, PyTorch, R Programming

Job Description

At AT&T, we’re connecting the world through the latest tech, top-of-the-line communications and the best in entertainment. Innovation powers all that we do, and our groundbreaking solutions deliver compelling communication and entertainment experiences to millions of customers worldwide. As part of our team, your next big idea could be our newest innovation. Here’s your opportunity to combine your passions for entertainment, media and technology to transform our company and your career.

The Big Data Software Engineer will be responsible for building big data pipelines using open source tools and enterprise cloud frameworks. This individual is expected to be hands-on and shall work closely with data scientists, data engineers and business SMEs to define and map data requirements into executable data processing pipes.

Key Roles and Responsibilities

Build high performing streaming and batch data pipelines that generate business ready datasets and insights to support existing use cases. Determine and document data mapping rules for movement of medium to high complexity data between applications, while adhering to and promoting the use of data administration standards. Support data selection, extraction, and cleansing for enterprise applications, including data warehouses and data marts. Investigate and resolve data issues across platforms and applications, including discrepancies of definitions, format, and function. Employee data mining techniques to achieve data synchronization, redundancy elimination, data reconciliation, quality control, and root cause analysis. Analyze existing PySpark/Scala/Java/Python/R code wherever necessary to evolve existing prototypes into modern scalable applications. Create reusable software components (ie specialized spark UDFs) and analytics applications. Support architecture evaluation of the enterprise data platform through implementation and launch of data preparation and data science capabilities. Act as the SME to support business needs in clear technical data transformation pipelines in collaboration with Data Engineers and AI Ops teams.

Qualifications

• Bachelors or Master's of Science in Computer Science, Math or Scientific Computing preferred.
• Requires 5-8 years experience.
• 6+ years of software development experience preferred: Java, Scala, Python, R with understanding of key functional programming principles.
• 3+ years of experience with Teradata, Hive, Snowflake, or other Big Data Warehouses.
• Prior experience with Big Data technologies, preferably Spark. Strong understanding and use of both relational and NoSQL databases.
• Experience with deploying complex data science solutions, especially NLP-based solutions, is a strong plus.
• Experience with data exploration, prototyping and visualization tools: e.g. zeppelin, Jupyter.
• Experience with high volume batch & streaming pipelines for disparate data sources would be a plus. Independent, self-starter, with demonstrated communication, collaboration and willing to learn passionately.

Job ID 2121516 

Date posted 05/04/2021