Azure Databricks, PySpark

Role & Responsibilities:

• Design, develop, and maintain robust data pipelines using Azure Databricks and PySpark
• Develop ETL/ELT workflows leveraging Azure Data Factory
• Write efficient PySpark queries and SQL for large-scale data processing
• Implement Medallion architecture for structured data pipelines
• Ensure data quality, integrity, and performance of pipelines
• Collaborate with business and technical stakeholders to gather requirements
• Lead a small team of data engineers and provide guidance on best practices
• Document technical designs, data flows, and processes
• Support CI/CD pipelines and DevOps practices for data solutions (preferred)
• Assist in data visualization and analytics initiatives (preferred)

Preferred Candidate Profile:

• Bachelor's degree in Computer Science, Information Technology, or related field
• Strong SQL skills and relational database knowledge
• Excellent problem-solving and analytical abilities
• Experience in Retail domain is a plus
• Familiarity with CI/CD pipelines and DevOps for data engineering
• Excellent communication and team collaboration skills