We are seeking a Senior Data Engineer with strong developer DNA someone who codes daily in PySpark, SQL, and Python, builds end-to-end data solutions, and leads design of robust, production-grade data platforms on Microsoft Fabric, Azure Databricks, and ADF. This is a hands-on leadership role for engineers who can think architecturally and code efficiently.
Key Responsibilities:
- Design, develop, and optimize complex data pipelines and lakehouse architectures.
- Lead development of batch and streaming ingestion frameworks using ADF, Databricks, and Event Hubs.
- Write optimized PySpark/SQL transformations for performance, scalability, and cost efficiency.
- Build reusable data frameworks and automation scripts for continuous integration and deployment.
- Establish unit testing, code review, and versioning standards across engineering teams.
- Integrate with DevOps CI/CD pipelines for automated releases and environment management.
- Mentor junior engineers on coding practices and architecture design.
- Collaborate with data architects on Fabric-based Lakehouse implementations.
Skills & Qualifications:
- 69 years of strong hands-on experience in data engineering and development.
- Expertise in:
- PySpark, SQL, Python, and performance tuning of large-scale transformations.
- Databricks, ADF, ADLS, Fabric Data Pipelines, and Delta Lake.
- Deep understanding of data architecture, schema design, job orchestration, and data governance.
- Strong knowledge of CI/CD (Azure DevOps) and infrastructure-as-code for data pipelines.
- Exposure to GCP BigQuery, Dataproc, or Dataflow as secondary platforms.
Preferred Certifications:
- Databricks Certified Data Engineer Professional
- Microsoft Certified: Fabric Analytics Engineer Associate (DP-600)
- Microsoft Certified: Azure Solutions Architect Expert (AZ-305)
- Google Cloud Certified Professional Data Engineer
