We are seeking an experienced
Data Engineer / Data Architect with strong expertise in
data architecture, data modeling, Databricks, PySpark, and Azure. The ideal candidate will design scalable data pipelines, optimize large-scale datasets, and implement robust ETL and data warehouse solutions while ensuring data governance, security, and compliance.
Key Responsibilities
- Design and implement data architectures, including star and snowflake schemas for analytics and reporting
- Build and maintain scalable data pipelines using Databricks, PySpark/Spark, and MySQL
- Develop and optimize Databricks notebooks and manage Unity Catalog for data governance and access control
- Process and optimize multi-terabyte datasets, ensuring high performance and cost efficiency
- Design and manage ETL/ELT workflows and data warehouse architectures
- Collaborate with stakeholders to translate business requirements into scalable data solutions
- Implement data governance, security, and compliance best practices
- Optimize data storage, compute costs, and query performance
- Monitor data quality, lineage, and reliability across pipelines
- Work closely with cross-functional teams including Data Science, Analytics, and Product teams
Required Skills & Experience
- 5+ years of experience in data architecture, data modeling, and database design
- Strong experience with star & snowflake schema design
- 3+ years of hands-on experience with Databricks, Unity Catalog, and PySpark/Spark
- Experience building scalable data pipelines using PySpark, Spark, and MySQL
- Hands-on experience working with large-scale (multi-terabyte) datasets
- Strong understanding of ETL processes and data warehouse architectures
- Familiarity with Azure cloud platform and related data services
- Experience with data governance, security, and compliance frameworks
- Strong SQL skills and database optimization expertise
- Excellent communication, analytical, and problem-solving skills
Good to Have
- Experience with Azure Data Factory, Azure Synapse, ADLS, or Delta Lake
- Exposure to Power BI / Tableau
- Experience with CI/CD for data pipelines
- Knowledge of ML pipelines or advanced analytics
Skills: azure,data governance,pipelines,datasets,data,etl,data warehouse
