Application Reliability Engineer – Data & AI
Search Jobs
Search by job, company or skills
Search by job, company or skills
- - - - - - - - - - - -
The Data and AI Application Reliability Engineer you are responsible for owning the post-deployment health, latency, and performance of productionized AI application and Medallion architectures followed for applications.
Acting as the problem-solver for technical issues affecting data pipelines, databases, and deployed AI/ML models, ensuring continuous operation and high user satisfaction.
Partner with core Feature teams to perform root-cause analysis, optimize PySpark jobs, and reduce system debt
Build automated observability dashboards and self-healing mechanisms for automated failure recovery
As a part of Job you are required to balance your responsibilities between Proactive Engineering & Automation and Operational Reliability/ Production Health.
Key Responsibilities
Analyze and refactor resource-intensive PySpark jobs, queries, and API endpoints to optimize cost, execution speed, and compute efficiency.
Develop automated recovery routines, DAG rerun triggers, and data quality checks to minimize manual intervention.
Partner closely with Feature Teams and Architects to establish strict Definition of Done (DoD) standards and production readiness gates for new deployments.
Design, implement, and maintain real-time monitoring and alerting frameworks for Medallion architecture pipelines, feature stores, and AI Applications.
Lead technical resolution for high-priority production incidents, conducting thorough post-mortems to eliminate recurring failure patterns.
Technical Expertise:
Azure Data Engineering Stack
Proficiency in Databricks , Python, and PySpark.
Azure (ADLS Gen2, Azure Data Factory, Key Vault, Azure DevOps)
Hands-on experience with Medallion architecture.
Cloud and DevOps Fundamentals
Understanding of cloud computing concepts and Services, specifically Microsoft Azure.
Good handson experience on Python.
Good to Have Technical Abilities :
Understanding of PowerBI reportswill be a plus.
DevOps & CI/CD Fundamentals
AI & Machine Learning Fundamentals
Data Science Fundamentals
Generative AI & LLM Fundamentals
Behaviour
Problem Solver:Ability to reverse-engineering complex system behavior and tracking down bugs.
Automation-First:An instinct to automate repetitive tasks .
Clear Communicator: Ability to explain technical root causes to non-technical stakeholders clearly.
Relevant work experience - 5 yrs
Job ID: 153100137
