Databricks Engineer

Job Role: Databricks Engineer

Total Experience Required: 8+ years

Job Location: Chennai, Bengaluru, Hyderabad

Job Mode: Hybrid

Key Skills:

• Strong Databricks and Databricks Studio knowledge
• Databricks Architecture [Is a plus]
• Apache Spark & PySpark
• Delta Lake & Databricks Workflows
• Azure / GCP Data Platforms
• Data Modeling & Lakehouse Architecture
• Unity Catalog & Data Governance
• Python & Large-Scale Data Processing
• Generative AI / AI Agents on Databricks

Roles & Responsibilities:

• Lead Databricks Architecture: Own the architecture, design, and implementation of enterprise-grade Databricks platforms, with primary ownership of Databricks Studio, Delta Lake, Workflows, and scalable data solutions.
• Drive Data Engineering & Performance: Design and optimize large-scale data processing using Apache Spark and PySpark, including cluster management, performance tuning, workload optimization, and cost efficiency.
• Define Cloud & Data Architecture: Architect solutions across Azure (ADLS, Synapse) and/or GCP (BigQuery, GCS), applying appropriate data modeling patterns including Dimensional, Data Vault, Canonical, and Lakehouse architectures.
• Lead Governance & Delivery: Establish governance, security, metadata, and access controls using Unity Catalog or equivalent frameworks, while leading complex cross-functional data initiatives and mentoring engineering teams.
• Drive AI/ML & Innovation: Evaluate and integrate AI agent frameworks and Generative AI use cases with Databricks, contributing to AI/ML lifecycle implementation and production-grade integration of data, models, and workflows.