Data Engineer

Key Skills

Aartificial intelligence, Apache Airflow, Apache Kafka, AWS, Azure, Big Data Technology, Data Analytics, Data Engineering, Data pipelines, Data science techniques, Database, Effective communication skills, ETL frameworks, Google Cloud Platform (GCP), Machine learning techniques, NumPy, Pandas, PyTorch, SPARK Programming

Job Description

We are launching a pioneering research and innovation hub in AI—one that will shape the way humans and machines collaborate. Led by Dr. Usama Fayyad, the Institute for Experiential AI (IEAI) is built around the challenges and opportunities made possible by human-machine collaboration. The institute provides a framework to design, implement, and scale AI-driven technologies in ways that make a true difference to society. Our ability to respond to the opportunities afforded to society will depend on training and building a workforce that is AI-capable. Located in Portland, Maine, with the goal of growing the Maine tech economy, the Roux Institute will be a center of activity for the Institute’s efforts.

Founded in 1898, Northeastern is a global research university and the recognized leader in experience-driven lifelong learning. Our world-renowned experiential approach empowers our students, faculty, alumni, and partners to create impact far beyond the confines of discipline, degree, and campus.

This role collaborates with researchers and data scientists from various fields including Applied Analytics, Bioinformatics, Biotechnology, Computer Science, Cybersecurity, Genomics, Health Data Analytics, Law & ethics, Neuroscience and Therapeutics discovery. General research and interpersonal skills are especially valued. We are building a new team together to deliver the true promise of AI, which lies at the intersection of humans and intelligent machines.

The Culture

Here at the Institute of Experiential AI (IEAI) we are committed to the highest of standards in all that we do. Working at the Institute of Experiential AI offers opportunities, an environment, a culture that just aren’t found together anywhere else. This is the right place for you if you’re curious, motivated by the future of technology and want to be part of a unique community that works on high-impact business and societal problems

Summary

In this role, you will be responsible for building out key components of data ingestion and transformation, using a combination of strong Java/Scala development skills as well as Big Data processing technologies. As a member of the team, you will be expected to take ownership of individual platform components and help set the vision and architecture for it. In the process, you will identify the requirements of new features, and propose design and drive the solution.

Minimum Requirements:

Education & Experience: Bachelor’s degree in Computer Science, Engineering, or similar disciplines and 5 or more years of professional experience, or can convincingly demonstrate this level of skill.

Knowledge and Skills:

• Expertise writing clear, maintainable, production-level code in Python.

• Experience with cloud platforms such as Azure, AWS, or Google Cloud.

• Experience with containerized application development with Docker or Podman.

• Expertise on ETL pipelines, data migration, data cleaning.

• Experience with big data pipelines (Apache Spark, Kafka, AirFlow) is a plus

• Experience with deep learning frameworks, such as Pytorch, is a plus

• Experience with machine learning frameworks such as SKLearn, AutoML is a plus

• Experience with vectors, matrix, dataframe libraries such as Numpy, Pandas is a plus

• Expertise on best practices for data governance, curation, validation, integrity.

• Expertise on version control best practices.

• Experience with relational and non-relational databases.

• Experience preparing well-documented, reproducible results for external stakeholders.

Preferred Experience:

• Expertise in setting up cloud architecture, particularly in AWS

• Experience working on a collaborative codebase with data science and engineering

• Data science, analytics, or machine learning.

• Experience within a particular domain such as healthcare, logistics, genomics.

• Open source ecosystem contributions or expertise.

Values & Abilities:

• Aptitude to independently learn new technologies, prototype and propose software design and solutions.

• Ability to communicate effectively across academia and industry.

• Team-player who can collaborative effectively across many teams within the University.

• Open-minded and assertive when collaborating and working within our team and with other groups within Northeastern University.

• Entrepreneurial mind-set with an ability to navigate complex structures and processes.

Key Responsibilities & Accountabilities:

40% - Build, maintain and improve data pipelines, infrastructure and tools to support Institute projects and operations in data science and AI

10% - Support data scientists and other end users such as faculty and postdocs

20% - Support delivery of practicum courses and other academic programs as subject matter expert

30% - Participate in delivery of projects at the AI Solutions Factory to solve AI problems of industry partners

Position Type

Information Technology

Additional Information

Northeastern University considers factors such as candidate work experience, education and skills when extending an offer. 

Northeastern has a comprehensive benefits package for benefit eligible employees. This includes medical, vision, dental, paid time off, tuition assistance, wellness & life, retirement- as well as commuting & transportation. Visit https://hr.northeastern.edu/benefits/ for more information.