BI Data Engineer

Key Skills

SQL, SQL Server, Apache Kafka, Spark Core, Scala Programming, Machine learning techniques, Data Visualization, ETL/ELT technologies, Data Analytics, SaaS, Thermal Power Plant, Databricks, Data Engineering, Data Lake, Data pipelines, Optimization, AzureML, DOMO, Data cleaning / Data cleansing

Job Description

As a Business Intelligence Data Engineer in our growing Business Analytics & Data Platform teams, you will support the development and maturing of a data-driven organization by creating intuitive means of accessing analytics by transforming data and developing, constructing, and testing cloud-based data architectures.

Collaborating with data scientists, data engineers, business analytics stakeholders, and analytics developers, you will optimize the databases you work with using your knowledge of data warehouse solutions, data modeling, and ETL. Although not a data scientist, you’ll be fully integrated into our data science team and the modeling process. You will serve as an internal resource for all things data, advising on data quality issues, feature creation, and deployment best practices.

Responsibilities:

• Build and maintain data systems and pipelines to transform legacy data structures distributed across dozens of cloud databases, on-prem databases, and a data warehouse into cleaned, well-organized, and documented data structures and data features ready for use by business analytics consumers and analysts
• Develop and maintain the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using primarily SQL Server, Databricks, and Azure technologies
• Use Python, Scala and SQL to assist with data cleaning, profiling, and exploration
• Collaborate with cross-functional business teams in Marketing, Sales, Operations, etc.

Qualifications:

• 3+ years in a data engineer role supporting data science and analytics teams, preferably in a SaaS company or other environment with web-scale data (billions of rows) and cloud model deployments
• Ability to thrive in a fast-paced remote-first startup environment
• Proactive and self-directed; excels at identifying opportunities to add value and contribute to the overall success of the team
• MS-SQL and Python
• Performance optimization
• Data warehouse technologies such as Microsoft Synapse
• Building and maintaining Data pipelines
• Experience using Databricks
• Proficiency with Azure Data Factory
• Working knowledge of reporting tools such as Tableau, Power BI (using DAX), Domo, etc
• Experience with Apache Spark, Hadoop, or other parallel processing technologies
• Experience with data visualization libraries, ER diagrams or tools
• Strong understanding of data transformation techniques and cloud data platforms
• Familiar with Data Lake and best practices
• Experience with Kafka Streaming (highly desirable)
• High-level familiarity with machine learning techniques and processes (highly desirable)