V
Azure Data Engineer
V
Azure Data Engineer
VirtusaThis job is no longer accepting applications
Job Description
You will work closely with clients, architects, and cross-functional teams to deliver robust data solutions that support analytics, reporting, and advanced data processing.
Design and implement end-to-end data engineering solutions on Microsoft Azure.
Develop and maintain data pipelines using Azure Data Factory (ADF).
Build scalable data processing frameworks using Azure Databricks (PySpark/Scala).
Architect and manage data lakes using Azure Data Lake Storage (ADLS Gen2).
Develop data warehousing and analytics solutions using Azure Synapse Analytics.
Work with HDInsight for big data workloads (Hive, Spark, Kafka, etc.) where applicable.
Optimize data workflows for performance, scalability, and cost-efficiency.
Implement data governance, security, and compliance best practices.
Collaborate with stakeholders to gather requirements and translate them into technical solutions.
Provide technical leadership and mentor junior team members.
Troubleshoot production issues and ensure high availability of data systems.
Required Skills & Qualifications
Core Technical Skills
Strong experience with
Azure Data Factory (ADF)
Azure Databricks (PySpark, Spark SQL)
Azure Data Lake Storage (ADLS Gen2)
Azure Synapse Analytics
HDInsight (Spark/Hadoop ecosystem)
Proficiency in Python and/or Scala
Strong SQL and data modeling skills (star/snowflake schema)
Experience with ETL/ELT design patterns
Knowledge of distributed data processing frameworks
Azure & Data Expertise
Hands-on experience with Azure-native data services
Understanding of data lake, lakehouse, and data warehouse architectures
Experience implementing CI/CD pipelines (Azure DevOps)
Familiarity with data security, RBAC, and encryption in Azure
Experience with monitoring tools (Azure Monitor, Log Analytics)
Nice to Have
Experience with Delta Lake / Lakehouse architecture
Exposure to real-time data processing (Kafka/Event Hubs)
Knowledge of Power BI or other visualization tools
Certifications such as
Microsoft Certified Azure Data Engineer Associate
Soft Skills
Strong problem-solving and analytical thinking
Excellent communication and stakeholder management skills
Ability to work in a fast-paced consulting environment
Leadership and mentoring capabilities
Roles & Expectations (T2 Senior Consultant Level)
Independently handle medium to large-scale projects
Design and implement end-to-end data engineering solutions on Microsoft Azure.
Develop and maintain data pipelines using Azure Data Factory (ADF).
Build scalable data processing frameworks using Azure Databricks (PySpark/Scala).
Architect and manage data lakes using Azure Data Lake Storage (ADLS Gen2).
Develop data warehousing and analytics solutions using Azure Synapse Analytics.
Work with HDInsight for big data workloads (Hive, Spark, Kafka, etc.) where applicable.
Optimize data workflows for performance, scalability, and cost-efficiency.
Implement data governance, security, and compliance best practices.
Collaborate with stakeholders to gather requirements and translate them into technical solutions.
Provide technical leadership and mentor junior team members.
Troubleshoot production issues and ensure high availability of data systems.
Required Skills & Qualifications
Core Technical Skills
Strong experience with
Azure Data Factory (ADF)
Azure Databricks (PySpark, Spark SQL)
Azure Data Lake Storage (ADLS Gen2)
Azure Synapse Analytics
HDInsight (Spark/Hadoop ecosystem)
Proficiency in Python and/or Scala
Strong SQL and data modeling skills (star/snowflake schema)
Experience with ETL/ELT design patterns
Knowledge of distributed data processing frameworks
Azure & Data Expertise
Hands-on experience with Azure-native data services
Understanding of data lake, lakehouse, and data warehouse architectures
Experience implementing CI/CD pipelines (Azure DevOps)
Familiarity with data security, RBAC, and encryption in Azure
Experience with monitoring tools (Azure Monitor, Log Analytics)
Nice to Have
Experience with Delta Lake / Lakehouse architecture
Exposure to real-time data processing (Kafka/Event Hubs)
Knowledge of Power BI or other visualization tools
Certifications such as
Microsoft Certified Azure Data Engineer Associate
Soft Skills
Strong problem-solving and analytical thinking
Excellent communication and stakeholder management skills
Ability to work in a fast-paced consulting environment
Leadership and mentoring capabilities
Roles & Expectations (T2 Senior Consultant Level)
Independently handle medium to large-scale projects
More Info
Key Skills
Azure Data Lake Storage ADLS Gen2
Log Analytics
encryption in Azure
distributed data processing frameworks
Azure-native data services
Azure Data Factory ADF
CI CD pipelines
ETL ELT design patterns
Azure Monitor
lakehouse
data warehouse architectures
About Company
Skills:
Azure Data Factory, Pyspark, Azure Data Lake, Azure Databricks, Sql
Skills:
Pyspark, Adf, Spark, Azure Databricks, Data Engineer, Azure, Synapse, ADLS, Data Bricks
Skills:
Azure Data Factory (ADF), Cloud Integration, Technology