We are looking for a Senior Databricks Administrator with strong expertise in Databricks on AWS, focused on security, governance, and cost optimization. This role will own platform administration, IAM-based data access controls, Unity Catalog governance, and infrastructure automation. The ideal candidate should demonstrate deep AWS IAM knowledge, production support maturity, and strong stakeholder management skills.

Key Responsibilities

Administer and manage Databricks workspaces (AWS E2 architecture)
Design secure S3 access using IAM roles, trust policies, and instance profiles
Implement fine-grained access control (FGAC) using Unity Catalog (catalogs, schemas, tables, system catalog)
Manage data governance policies, storage credentials, and external locations
Define and enforce cluster policies for security and cost control
Monitor cluster utilization, implement autoscaling, and drive cost optimization (DBUs, Spot instances)
Provision and manage Databricks infrastructure using Terraform or Asset Bundles (IaC)
Troubleshoot production incidents related to IAM, storage, and clusters
Build and maintain dashboards within Databricks for monitoring and insights
Collaborate with security, infrastructure, and business teams

Required Skills

5+ years of Databricks administration on AWS
Strong hands-on experience with AWS IAM (roles, trust relationships, external ID, STS, instance profiles)
Good understanding of AWS networking basics (VPC, Route53)
Strong experience with Unity Catalog and governance models (including system catalog)
Experience with cluster policy management and cost optimization strategies
Strong scripting skills in Python or Bash
Experience with Infrastructure as Code (Terraform / Databricks Asset Bundles)
Experience building dashboards in Databricks (SQL / Lakehouse monitoring)
Proven experience supporting production environments and stakeholder communication

Preferred

Experience with multi-workspace governance and centralized metastore
Exposure to chargeback/showback models
AWS Certification (Solutions Architect / Security)

AWS Admin

AWS Databricks Admin