Job Description
The ADL / Databricks Platform Operations Engineer is responsible for providing business-as-usual (BAU) operational support for the Azure Data Lake (ADL) and Databricks platforms, with a primary focus on IST time zone coverage. This role is a key part of the U.S. ADL Platform Operations team and enables a support model to ensure platform availability, stability, and responsiveness.
Responsibilities
- Provide operational support for Azure Data Lake and Databricks platforms during IST hours, ensuring continuity of support outside U.S. business hours.
- Maintain platform health across environments (DEV, QA, UAT, PROD); triage, investigate, and resolve operational issues or escalate as appropriate.
- Support BAU operational activities, including incident management, service requests, platform readiness checks, and ongoing operational maintenance.
- Act as a liaison between platform engineering, data engineering teams, and business stakeholders, ensuring alignment on operational priorities, support needs, and platform usage.
- Advise teams on ADL and Databricks operational best practices, including stability, performance, security, and efficient platform usage.
- Apply a high-level understanding of the full data stack (ingestion storage compute orchestration serving) to diagnose issues, assess downstream impact, and support reliable data delivery.
- Support and troubleshoot modern data hydration patterns, including incremental loads, CDC-based pipelines, backfills, and data freshness strategies, with an awareness of cost and performance tradeoffs.
- Monitor and improve platform observability through logging, metrics, alerting, and performance diagnostics; identify bottlenecks and proactively improve platform reliability.
- Ensure operational processes align with enterprise data quality, security, and governance standards.
- Identify recurring issues, operational gaps, and opportunities for process improvement or automation to enhance platform reliability and support efficiency.
- (Valuable but not mandatory) Contribute a system design mindset by considering scalability, reliability, and maintainability when evaluating operational issues or proposing improvements.
Qualifications
- Bachelor's degree in Computer Science, Engineering, Information Systems, or a related technical field, or equivalent practical experience.
- Experience supporting Databricks or Azure-based data platforms in a production operations role.
- Hands-on experience with Azure services, including Azure Data Lake, Azure Data Factory, and/or Databricks.
- End-to-end cloud infrastructure understanding, with the ability to reason about how data platform components interact across the full data lifecycle.
- Experience with incident management, operational monitoring, and production support in an enterprise environment.
- Familiarity with industry best practices for data pipeline operations, including incremental processing, CDC, backfills, and data freshness management.
- Experience with system health monitoring, logging, alerting, and performance troubleshooting in distributed data platforms.
- Strong troubleshooting and analytical skills, with the ability to resolve complex operational issues.
#KGS
