Senior Data Engineer

Primary Responsibilities:

Data Engineering & Development

Design, code, test, document, and maintain high-quality, scalable data pipelines and solutions in cloud environments

Ingest and transform data from various sources including APIs, streaming platforms, files, and databases

Develop reusable patterns and promote innovation to enhance team productivity and velocity

Application Development & Deployment

Design and develop applications in an Agile environment

Implement and manage CI/CD pipelines for seamless deployment

Participate in prototyping, design reviews, and code reviews

Operations & Support

Willingness to work in both development and operations roles

Flexibility to support operations with adaptable working hours

Assist with incident and problem management

Collaboration & Leadership

Act as a self-starter with a proactive learning attitude

Serve as a resource and mentor to team members

Work independently on complex and less structured issues

Comply with the terms and conditions of the employment contract, company policies and procedures, and any and all directives (such as, but not limited to, transfer and/or re-assignment to different work locations, change in teams and/or work shifts, policies in regards to flexibility of work benefits and/or work environment, alternative work arrangements, and other decisions that may arise due to the changing business environment). The Company may adopt, vary or rescind these policies and directives in its absolute discretion and without any limitation (implied or otherwise) on its ability to do so

Technical Responsibilities

Data Architecture & Engineering

Design and implement Data Warehouses and Data Lakes using platforms like Databricks and Snowflake

Perform data modeling and architecture design

Ensure high standards of data quality and integrity

Cloud & Platform Engineering

Work with Platform-as-a-Service (PaaS) and cloud solutions, focusing on data stores and associated ecosystems

Manage design services, provide sizing and configuration assistance

Database & Storage

Engage in database architecture, optimization, security, and administration

Work with both NoSQL and relational databases

Write SQL and data markup scripts to support analytics and application development

Vendor & Business Analysis

Manage relationships with software and hardware vendors

Analyze current business practices and identify future opportunities for data leverage

- Undergraduate degree or equivalent experience.

Required Qualifications:

3+ years of experience with:

Design and implementation of Data Warehouse/Data Lake (Databricks/Snowflake)

Data architecture and data modeling

2+ years of experience with:

Databricks, Python, Spark, PySpark, SQL

Azure Data Factory

1+ years of experience with:

CI/CD tools (GitHub Actions, Jenkins, or similar)

Cloud-based CI/CD pipelines

GitHub

NoSQL and relational databases

Preferred Qualifications:

Experience with Apache Kafka

Knowledge of Data ingestion from various APIs

Working in Agile/Scrum environments