V
Lead Software Engineer
V
Lead Software Engineer
Virtusa- Posted 2 hours ago
- Be among the first 10 applicants
Job Description
'We are seeking a Databricks Data Engineer to design, develop, and optimize scalable data pipelines supporting enterprise analytics, AI/ML, and Agentic AI solutions.
The ideal candidate should have strong hands-on experience with Azure Databricks, Apache Spark (PySpark), Spark SQL, and Delta Lake for building batch and streaming data pipelines.
Experience with Azure Data Lake Storage, Unity Catalog, Kafka/Event Hubs, and data integration patterns is essential. The role requires expertise in ETL/ELT development, Lakehouse architecture, and data quality best practices.
Candidates should be proficient in Python, SQL, Git-based CI/CD, and Terraform.
Experience with Azure cloud services and performance tuning of Spark workloads is highly desirable.
The engineer will collaborate with data scientists, MLOps engineers, and platform teams to deliver reliable, governed, and scalable data solutions.
Strong analytical, troubleshooting, and communication skills are essential in a collaborative enterprise environment.
The ideal candidate should have strong hands-on experience with Azure Databricks, Apache Spark (PySpark), Spark SQL, and Delta Lake for building batch and streaming data pipelines.
Experience with Azure Data Lake Storage, Unity Catalog, Kafka/Event Hubs, and data integration patterns is essential. The role requires expertise in ETL/ELT development, Lakehouse architecture, and data quality best practices.
Candidates should be proficient in Python, SQL, Git-based CI/CD, and Terraform.
Experience with Azure cloud services and performance tuning of Spark workloads is highly desirable.
The engineer will collaborate with data scientists, MLOps engineers, and platform teams to deliver reliable, governed, and scalable data solutions.
Strong analytical, troubleshooting, and communication skills are essential in a collaborative enterprise environment.
More Info
Key Skills
Unity Catalog
performance tuning of Spark workloads
Azure Data Lake Storage
Event Hubs
Delta Lake
data quality best practices
Lakehouse architecture
Git-based CI/CD
About Company
Skills:
Aws Lambda, Graphql, Docker, Cloudformation, Rest Apis, Microservices, AWS CDK, CI CD pipelines, NoSQL databases, Python 3.x
Skills:
RDBMS, Rest Api, Java, Microservices, Devops, Oauth, Hibernate, Kafka, Kubernetes, Nosql, Azure, Graphql, Springboot, Docker, Spark, Infrastructure as Code
Skills:
Java, Springboot, CSS, Node.js, Json, Sql, HTML, Angular, Nosql, React, Gcp, Javascript, Typescript, Restful Apis, Azure, AWS, WebSockets, AI-assisted software development tools
Skills:
software design patterns , secure coding , Kubernetes, Rest Api, Browser Compatibility, Solid Principles, Typescript, React, Linux, Javascript, Unix Scripting, Pl Sql, Autosys, Automated Testing, Github, Accessibility, Gradle, Bdd, Cucumber, Oracle Sql, Docker, Jest, AI-assisted development tools, React Testing Library, enterprise integration patterns, frontend performance optimization, DevOps practices, test automation frameworks, build tooling
Skills:
Typescript, Sql, Java, HTML, CSS, Json, React, AWS, Angular, Node.js, Nosql, Azure, Springboot, Gcp, Javascript, Restful Apis, AI-assisted software development tools, WebSockets