Airflow Platform Engineer1
Experience - 4 - 6 years
Key Responsibilities
DAG Development & Developer Enablement
- Design, develop, and optimize Airflow DAGs for enterprise use cases
- Demonstrate strong experience with operators such as PythonOperator, SQL operators,
API/HTTP operators, KubernetesPodOperator etc.
- Apply advanced DAG patterns including dynamic DAGs, branching, sensors, retries, and
error handling
- Troubleshoot DAG failures, performance issues, and integration challenges
- Enable orchestration of modern data workflows including DBT pipelines
- Provide reusable templates, best practices, and onboarding support to drive self-service
adoption
Platform Operations & Kubernetes Support
- Support Airflow platform deployed on AKS, covering scheduler, webserver/API, workers,
and metadata services
- Troubleshoot issues related to task execution, scheduling delays, and resource bottlenecks
- Diagnose Kubernetes-level issues involving pods, networking, storage, and RBAC
- Work with Azure services such as Key Vault, Storage, and networking
- Support CI/CD pipelines for DAG and container-based deployments
- Improve platform stability, monitoring, and alerting
Required Skills
- Strong experience with Apache Airflow (3.x or later) in production environments
- Proven experience in DAG development using operators and orchestration patterns
- Working knowledge of Kubernetes (AKS preferred) to support Airflow runtime
- Expert-level Python knowledge for building, debugging, and maintaining production-
grade workflows and supporting libraries
- Expertise in writing unit, integration, and DAG validation tests for Airflow workflows,
including mocking operators, validating dependencies, and ensuring reliability in CI/CD
pipelines
- Experience with DBT and modern data pipelines
- Experience with CI/CD tools (Git, Jenkins, Docker)
- Familiarity with Azure or other cloud platforms
Preferred Skills
- Experience with Astronomer / Astro Private Cloud
- Exposure to data platforms such as Snowflake, and APIs
- Experience with monitoring tools such as Splunk or Prometheus
- Knowledge of secrets management using Azure Key Vault
Summary
This role is critical to ensuring reliable platform operations and scalable adoption of the
Airflow-based orchestration platform. The ideal candidate combines strong Airflow
expertise, Kubernetes knowledge, and hands-on development experience to support both
platform stability and developer productivity.
