DataBricks Technical Lead (Exp: 8 Yrs to 12 Yrs)

bilvantis technologies

8-12 Years

Not Disclosed

Hyderabad, India

Early Applicant

Posted 13 hours ago
Be among the first 10 applicants

Job Description

Job Description: DataBricks – Technical Lead (Exp: 8 Yrs to 12 Yrs)

We are seeking a talented and motivated DataBricks expert to join our team as a Technical Lead.

Required Skills & Qualifications

8 to 12 years of IT experience, with 3+ years focused on big data and Databricks.
Strong hands-on experience with Databricks, PySpark, and Spark SQL.
Experience with modern cloud data lakes (Azure ADLS, AWS S3).
Hands-on experience with Azure Data Factory (ADF) or AWS Glue for orchestration.
Strong experience with relational databases and SQL.
Strong background in Agile delivery, CI/CD pipelines, and version control (Git).
Experience with Delta Lake, Unity Catalog, and the medallion architecture.
Experience with streaming data (Structured Streaming, Kafka, Event Hubs).
Experience with Infrastructure-as-Code (Terraform).
Experience in agile development processes using Jira and Confluence.
Design scalable data engineering solutions and lead their delivery on Databricks.
Understanding of the SDLC and Agile methodologies.
Communication with the customer and producing the daily status report.
Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
Databricks Certified Data Engineer Professional certification.
Microsoft Certified: Fabric Data Engineer Associate (DP-700) or AWS Certified Data Engineer - Associate.
Should have good oral and written communication.
Should be able to lead and mentor a team.
Should be proactive and adaptive.

Key Responsibilities

Design scalable, high-performance data pipelines and Lakehouse workflows using Databricks and PySpark / Spark SQL.
Build complex ETL/ELT pipelines on cloud platforms such as Azure or AWS.
Guide development teams, perform code reviews, and enforce engineering best practices.
Mentor engineers on Databricks, Spark, and data engineering design patterns.
Work with product owners, business analysts, and architects to translate requirements into technical solutions.
Monitor production workloads, fine-tune ETL/ELT performance, and ensure data reliability.