Data Engineer

Key Skills

Amazon Elastic Compute Cloud-EC2, Apache Airflow, AWS, C Programming, Data Lake, Database, Java Programming, Large scale data processing, PySpark, Python Programming, RStudio, SAS, SQL, Tableau, Unix Operating system

Job Description

Data Science is at the core of Nielsen’s business. Our team of researchers come from diverse disciplines and they drive innovation, new product ideation, experimental design and testing, complex analysis and delivery of data insights around the world. We support all International Media clients and are located where our clients are.

DESCRIPTION

• Conduct one-on-one meetings with the manager and team meetings to gather the information needed for developing new software or code. Analyze methodology documents to understand client requirements.
• Design and develop software and related database infrastructure that will feed into the media data lake (ecosystem) through Nielsen’s data flow processes.
• Conduct code review with senior developers before pushing the code into production. Support analytics performed by the Data Science team by creating or modifying and then deploying and supporting software applications and data infrastructure to respond to custom analytics requests and client inquiries, and incorporate changing analytical and statistical methodologies,
• standards and best practices. Combine software engineering expertise with knowledge of Nielsen TV measurement tools and services.
• Create job flow, automate the jobs and run jobs. Implement data pipeline Directed Acyclic Graphs (DAGs) and maintenance DAGs. Configure and setup DAGs based on the data to run Spark commands in parallel and sequential. Create hooks,
• as well as Python, Bash, Spark and custom operators. Schedule a pipeline to run the DAG to fetch data for loading into a database.
• Collaborate with cross-functional Nielsen Media teams to develop and implement customized software solutions that will validate enhancements to the universe estimate methodologies. Perform unit testing using test cases and fix any bugs.

Write and run test scripts for Media and Annual Estimate data. Run test scripts for data generated from team members. Validate the local Return Path Data Universe Estimates (RPD UEs). Collaborate with Data Scientists by leveraging software engineering expertise to identify and address software development and data quality issues, and detect and resolve quality escapes. Perform detailed analysis of data quality issues, report to other teams on significant data changes, and rectify the problem. Respond to internal data analytics inquiries by determining how to execute the analyses with the platforms and applications available, modifying specifications as needed, providing an estimated delivery date, and accurately delivering data-driven analyses on time. Oversee (without supervisory authority) 2-3 data scientists. Involves domestic travel, 1-2 times/year for 1-2 days/trip. Involves the opportunity to telecommute from within the Oldsmar, FL area for up to 3 days/week, as feasible. Tools used: Python, PySpark, SQL, Databricks, Intelligence Studio, EC2, Airflow and Tableau.

QUALIFICATIONS

• Minimum requirements: Bachelor’s degree in computer science, engineering or a related field with an information technology focus plus 2 years of experience in software design,
• development and testing. This must include 2 years of experience in/with: solving complex design and coding problems;
• SQL and large-scale databases; UNIX; writing Python code, and converting SAS and/or C code to Python code;
• creating and loading tables in the Nielsen Media Data Lake, and migrating tables from 1.0 to 2.0; creating RPD (Return Path Data) universe estimates, and updating and maintaining the methodology with any new additions;
• and producing cluster analyzes for exclusion of Broadband Only UEs and RPD. 1 year of experience in/with:
• creating S3 buckets in the AWS environment to store and retrieve files, using Iterative Proportional Fitting (IPV);
• and designing databases. Willingness to travel domestically 1-2 times/year for 1-2 days/trip.