Data Engineer (3-6 Years)
Title: Data Engineer - Azure & Databricks
Responsibilities
- Design, develop, and maintain ADF pipelines with parameterization and error handling
- Write and optimize SQL queries for analytical and reporting workloads
- Develop Databricks notebooks using PySpark for ingestion and transformation
- Implement incremental data loads and reprocessing strategies
- Design and manage data structures in ADLS across multiple layers
- Develop production-ready Python code for data engineering workflows
- Implement data quality validations and reconciliation logic
- Investigate and resolve pipeline failures and data issues
- Collaborate with architects, analysts, and business stakeholders
Required Skills
- Strong proficiency in SQL
- Solid hands-on experience with Azure Data Factory
- Strong working knowledge of Databricks and PySpark
- Experience designing and managing data in ADLS
- Proficiency in Python for data engineering use cases
- Understanding of data modeling and analytical concepts
