Job Description:

We are looking for an experienced Azure Databricks Administrator to manage operate and optimize Azure Databricks platforms within an enterprise Azure ecosystem
The role is responsible for ensuring high availability security performance and cost optimization of Azure Databricks environments while supporting data engineering analytics and AI ML workloads
The ideal candidate will have strong hands on experience with Azure Databricks Azure infrastructure security and governance and will work closely with cloud data and business teams to deliver a stable and scalable data platform

Key Responsibilities:

Administer and manage Azure Databricks workspaces across Development QA and Production environments
Configure and manage interactive and job clusters including autoscaling auto termination cluster pools and policies
Perform environment setup workspace configuration and platform upgrades
Ensure high availability and operational stability of Azure Databricks services
Integrate Azure Databricks with Azure Data Lake Storage Gen2 ADLS Azure SQL Synapse Event Hub and other Azure services
Manage VNet injected Databricks workspaces private endpoints and network security configurations
Work with Azure networking teams to ensure secure and optimized connectivity
Manage users groups and access controls using Azure Active Directory Entra ID
Implement data governance using Unity Catalog including catalog schema table and column level security
Configure secure authentication using AAD SSO Managed Identities and Service Principals
Manage secrets and credentials using Azure Key Vault integration
Ensure compliance with enterprise security audit and regulatory requirements
Monitor Azure Databricks usage and implement cost optimization strategies including
o Cluster right sizing
o Autoscaling and auto termination
o Instance pool configuration
Analyze workload patterns and recommend performance tuning best practices
Provide usage and cost reports to stakeholders and FinOps teams
Deploy and manage Azure Databricks infrastructure using Infrastructure as Code tools such as Terraform or ARM templates
Support CI CD pipelines for Databricks notebooks jobs libraries and workspace artifacts using Azure DevOps or GitHub
Automate operational tasks to reduce manual effort and improve reliability
Set up monitoring and alerting using Azure Monitor Log Analytics and Databricks metrics
Troubleshoot cluster failures job issues workspace access problems and performance bottlenecks
Perform root cause analysis and implement preventive measures
Provide Level 2 Level 3 support within a 24 7 operational model
Create and maintain platform documentation runbooks SOPs and architecture diagrams
Collaborate with data engineers data scientists cloud engineers and security teams
Participate in platform reviews and continuous improvement initiatives

Technical Requirements:

Mandatory Technical Skills
7 years of experience in data platform or cloud administration with strong focus on Azure Databricks
Hands on experience with Azure services including ADLS Gen2 Azure Networking Azure Monitor and Azure AD
Strong understanding of Apache Spark architecture and performance tuning
Experience managing secure enterprise scale Azure Databricks environments
Proficiency in Python PowerShell or Bash scripting
Hands on experience with Terraform or ARM templates
Preferred Good to Have
Experience with Unity Catalog data lineage and governance frameworks
Familiarity with Delta Lake job orchestration and workload isolation
Experience with CI CD pipelines using Azure DevOps or GitHub Actions
Understanding of FinOps practices in Azure
Certifications
o Azure Databricks Certified Associate Professional
o Microsoft Azure Administrator Solutions Architect
Soft Skills
Strong analytical and troubleshooting capabilities
Excellent communication skills with technical and business stakeholders
Ability to work independently in a fast paced 24 7 support environment
Proactive detail oriented and focused on operational excellence
Nice to Have
Experience supporting large enterprise or multi workspace Azure Databricks platforms
Exposure to onsite offshore delivery models and global teams
Experience working in regulated environments Healthcare

Additional Responsibilities:

Infosys is a global leader in next generation digital services and consulting
We enable clients in more than 50 countries to navigate their digital transformation
With over four decades of experience in managing the systems and workings of global enterprises we expertly steer our clients through their digital journey
We do it by enabling the enterprise with an AI powered core that helps prioritize the execution of change
We also empower the business with agile digital at scale to deliver unprecedented levels of performance and customer delight
Our always on learning agenda drives their continuous improvement through building and transferring digital skills expertise and ideas from our innovation ecosystem
Infosys provides equal employment opportunities to applicants and employees without regard to race color sex gender identity sexual orientation religious practices and observances national origin pregnancy childbirth or related medical conditions status as a protected veteran or spouse family member of a protected veteran or disability

Preferred Skills:

Technology->Infrastructure-Server Administration->Cloud Infrastructure->Azure Administration