Job Title: Azure Data Engineer (Hybrid)
Location: Chennai
Experience: 7+ Years
Job Summary
We are seeking an experienced
Azure Data Engineer to design, implement, and manage scalable, high-performance data solutions on Microsoft Azure. The ideal candidate will have strong expertise in Azure Synapse Analytics, PySpark, and related Azure services, with proven experience in building and optimizing data pipelines, transformations, and integrations.
Roles & Responsibilities
- Design, develop, and manage scalable data solutions using Azure Synapse Analytics and PySpark.
- Develop and optimize Synapse Pipelines, Data Flows, and data integration processes.
- Process and transform large datasets with PySpark integrated into Synapse Analytics.
- Integrate solutions with Azure Data Lake Storage, Azure Data Factory, Azure Databricks, and Azure SQL Database.
- Create and fine-tune SQL queries (Synapse SQL & Spark SQL) for analytics and performance optimization.
- Build automated workflows and solutions using PowerShell or other scripting tools.
- Ingest data from multiple sources, including cloud storage and on-premises systems.
- Ensure efficient database design aligned with best practices for normalization and performance.
- Troubleshoot and resolve data pipeline issues, transformation errors, and workflow disruptions.
- Contribute to data migration projects, including Oracle to PostgreSQL migrations.
Required Skills
- Strong hands-on experience with Azure Synapse Analytics, PySpark, and Azure Data Factory.
- Proficiency in Spark SQL and T-SQL for performance optimization.
- Experience with Azure Databricks and Azure Data Lake Storage.
- Solid understanding of data modeling, normalization, and performance tuning.
- Familiarity with PowerShell or similar scripting for automation.
- Experience with data ingestion from diverse sources.
- Knowledge of database migration best practices, especially Oracle to PostgreSQL.
Good to Have
- Knowledge of Azure DevOps for CI/CD pipeline implementation.
- Experience in implementing data security and governance on Azure.
- Exposure to big data analytics frameworks.
