Senior AI Data Engineer

Senior Manager

Job Location: Bengaluru/ Delhi NCR

Shift Timing: 1:00PM IST – 10:00 PM IST

Experience: 5+ years

Job Responsibilities:

• Design and build scalable, production-grade data engineering solutions using the Databricks Lakehouse Platform
• Develop client-focused Proof of Concepts (POCs) demonstrating modern data engineering and analytics capabilities
• Translate business and analytical requirements into robust data models and layered architectures (Bronze/Silver/Gold)
• Design robust data ingestion, transformation, and orchestration pipelines using Databricks best practices
• Implement enterprise-grade data governance, security, and access control using Unity Catalog and related Databricks capabilities
• Stay current with new Databricks product capabilities and recommend innovative approaches for solving client challenges
• Collaborate with cross-functional teams including Data Scientists, AI Engineers, Cloud Engineers, and Business Consultants
• Mentor junior engineers and contribute to developing reusable solution accelerators and best practices

Must have:

• 5+ years of experience in Data Engineering
• Expert knowledge of the Databricks platform and its ecosystem, with prefer
• Demonstrated ownership of cost optimization for Databricks workloads, including cluster policies, workload isolation, Photon usage, job vs. interactive compute, and SQL/query cost management
• Exposure to Generative AI, LLMs, AI Agents, or AI-assisted data engineering
• Strong hands-on experience with Databricks Lakehouse Architecture, Delta Lake, Unity Catalog, Databricks SQL, Spark SQL, and PySpark.
• Deep understanding of data modeling, distributed computing, and modern data architecture principles
• Experience building scalable ETL/ELT pipelines for enterprise workloads
• Hands-on experience with workflow orchestration using Databricks Workflows, Airflow, or similar orchestration platforms
• Expertise in data governance, security, and access controls, including Unity Catalog, RBAC, data masking, and secure data sharing
• Experience working on at least one major cloud platform (Azure, AWS, or GCP).
• Experience integrating AI capabilities into enterprise data platforms.
• Excellent analytical, problem-solving, and communication skills, with the confidence to engage directly with enterprise clients and business stakeholder
• Ability to independently own solution design and rapidly develop Proof of Concepts

Good to Have:

• Databricks certifications
• Knowledge of vector databases, RAG architectures, or AI orchestration frameworks
• Experience with CI/CD, Git, DevOps, and Infrastructure as Code
• Exposure to Power BI, Tableau, or other BI platforms.

Eligibility:

• Master's or Bachelor's degree in Math, Statistics, Economics, Computer Science or related analytics field from top-tier universities with strong record of achievement
• Minimum 5 years of relevant experience in Data Engineering
• Demonstrated expertise in designing and implementing enterprise-scale solutions on Databricks