Deep experience with: o Databricks Lakehouse architecture o Delta Lake o Unity Catalog o MLflow
Strong understanding of batch and streaming data architectures
Experience designing multi-workspace or multi-tenant setups Agentbricks & AI Knowledge
Hands-on or strong working knowledge of Agentbricks
Understanding of: o AI agent concepts (tools, memory, reasoning loops) o RAG architectures and vector search o LLM-driven workflows over enterprise data
Experience integrating agents with data, APIs, and downstream systems Data Engineering & Programming
Strong SQL expertise
Proficiency in Python and PySpark
Experience with ETL/ELT and data orchestration patterns
Familiarity with APIs and event-driven integration Databricks & Lakehouse Architecture
Define and own end-to-end Databricks Lakehouse architecture
Design scalable platforms for: o Structured, semi-structured, and unstructured data o Analytical and AI workloads
Establish architecture patterns, best practices, and reference designs
Lead modernization from legacy data platforms to Databricks-based lakehouse Agentbricks & AI Agent Enablement
Architect and enable AI agent-based solutions using Agentbricks
Design data and system patterns to support: o AI agents reasoning over enterprise data o Tool-calling and action-oriented agents o Retrieval-Augmented Generation (RAG) workflows
Define how agents interact with: o Delta tables o Vector embeddings o Feature stores and semantic layers
Guide teams on agent orchestration, memory, observability, and safety Advanced Analytics & AI Platform Design
Enable data architectures that support: o Machine learning pipelines o Feature engineering and reuse o Real-time and batch inference
Integrate Databricks with: o LLM providers and model endpoints o Vector search and embedding storage
Collaborate with data scientists and AI engineers to productionize AI solutions Databricks Platform Components
Architect and govern solutions using: o Delta Lake o Unity Catalog (governance, lineage, access control) o Databricks Workflows and Jobs o MLflow (experiments, models, lifecycle)
Define cluster, compute, and workspace strategies for multi-team environments Security, Governance & Compliance
Implement secure data and AI access using: o Unity Catalog permissions o Cloud IAM integrations
Define governance for: o AI agent data access o Model and agent versioning o Auditability and traceability
Ensure compliance with enterprise security and regulatory requirements Performance, Cost & Operations
Design for performance optimization and scalability
Optimize compute usage, storage, and agent execution costs
Enable monitoring and observability for: o Data pipelines o ML workloads o AI agents
Partner with platform and FinOps teams on cost control strategies Technical Leadership & Collaboration
Act as the Databricks and AI architecture authority
Review solution designs and guide engineering teams
Mentor data engineers, ML engineers, and architects
Translate business and AI requirements into scalable technical designs
Knowledge of architectural design patterns, performance tuning, database and functional designs
Hands-on experience in Service Oriented Architecture
Ability to lead solution development and delivery for the design solutions
Experience in designing high level and low level documents is a plus
Good understanding of SDLC is a pre-requisite
Awareness of latest technologies and trends
Logical thinking and problem solving skills along with an ability to collaborate
