Role OverviewWe are looking for a Senior Data Engineer who will play a key role in designing, building, and maintaining data ingestion frameworks and scalable data pipelines. The ideal candidate should have strong expertise in platform architecture, data modeling, and cloud-based data solutions to support real-time and batch processing needs.
What you'll be doing:- Design, develop, and optimiseDBT modelsto support scalable data transformations
- Architect and implementmodern ELT pipelinesusingDBTand orchestration tools likeApache Airflow and Prefect
- Leadperformance tuning and query optimizationfor DBT models running onSnowflake, Redshift, or Databricks
- IntegrateDBT workflows & pipelineswithAWS services(S3, Lambda, Step Functions, RDS, Glue) and event-driven architectures
- Implement robust data ingestion processes from multiple sources, including manufacturing execution systems (MES), Manufacturing stations, and web applications
- Manage and monitororchestration tools(Airflow, Prefect) for automated DBT model execution
- ImplementCI/CD best practicesfor DBT, ensuringversion control, automated testing, and deployment workflows
- Troubleshootdata pipeline issuesand provide solutions for optimizingcost and performance
What you'll have:- 5+ years of hands-on experience with DBT, includingmodel design, testing, and performance tuning
- 5+ years of Strong SQL expertise with experience inanalytical query optimizationanddatabase performance tuning
- 5+ years of programming experience, especially inbuilding custom DBT macros, scripts, APIs, working with AWS services using boto3
- 3+ years of Experience withorchestration toolslikeApache Airflow, Prefectfor scheduling DBT jobs
- Hands-on experience inmodern cloud data platformslikeSnowflake, Redshift, Databricks, or Big Query
- Experience withAWS data services(S3, Lambda, Step Functions, RDS, SQS, CloudWatch)
- Familiarity with serverless architectures and infrastructure as code (CloudFormation/Terraform)
- Ability to effectively communicate timelines and deliver MVPs set for the sprint
- Strong analytical and problem-solving skills, with the ability to work across cross-functional teams
Nice to haves:- Experience in hardware manufacturing data processing
- Contributions to open-source data engineering tools
- Knowledge of Tableau or other BI tools for data visualization
- Understanding of front-end development (React, JavaScript, or similar) to collaborate effectively with UI teams or build internal tools for data visualization
