Position Overview
Job Title: QA Automation Engineer / Senior QA Automation Engineer (AI Platforms), AVP
Location: Pune, India
Role Description
The QA Automation Engineer ensures quality, reliability, and stability of AI-powered applications and platforms. The role involves designing automation frameworks, validating machine-learningdriven outputs, and ensuring consistent performance of both traditional and AI-driven systems. You will work with developers, data scientists, ML engineers, and product teams to build robust test strategies that validate model behavior, data quality, and AI user experiences.
What We'll Offer You
As part of our flexible scheme, here are just some of the benefits that you'll enjoy
- Best in class leave policy
- Gender neutral parental leaves
- 100% reimbursement under childcare assistance benefit (gender neutral)
- Sponsorship for Industry relevant certifications and education
- Employee Assistance Program for you and your family members
- Comprehensive Hospitalization Insurance for you and your dependents
- Accident and Term life Insurance
- Complementary Health screening for 35 yrs. and above
Your Key Responsibilities
Core QA Responsibilities
- Design and maintain automation frameworks for UI, API, and backend testing.
- Develop test automation scripts using Selenium, Playwright, Cypress, or similar.
- Build API automation suites using RestAssured, Postman/Newman, etc.
- Perform test planning, execution, reporting, and defect lifecycle management.
- Implement automated tests in CI/CD pipelines (Jenkins, TeamCity, GitHub Actions).
- Perform root-cause analysis and support L3 production issues.
AI-Focused QA Responsibilities
- Collaborate with Data Science and ML teams to test AI/ML model behavior.
- Validate model outputs for accuracy, precision, recall, thresholds, and stability.
- Test AI-driven features: recommendations, NLP/chatbots, classification, anomaly detection, predictive insights, etc.
- Automate validation of model inference APIs, streaming outputs, and batch pipelines.
- Conduct data-quality testing for ML feature inputs (schema, drift, distribution checks).
- Validate model retraining, deployment workflows, and versioning pipelines.
- Test UI interfaces showing model predictions, confidence scores, and explainability outputs.
- Ensure fairness, bias detection, and responsible AI presentation in product workflows.
- Collaborate with MLOps to test end-to-end ML lifecycle.
Tools & Frameworks for AI Model Testing
- AI/ML Testing Frameworks
- Deepchecks ML model validation, data integrity, drift checks
- DeepEval / TruEra / Arthur AI model evaluation, quality monitoring, bias & fairness checks
- Evidently AI automated monitoring for model drift, data drift, data quality
- Fiddler AI model explainability, fairness, performance dashboards
- MLflow model validation, experiment tracking, test comparison
- Great Expectations data validation for ML pipelines
Data & Pipeline Testing
- PyTest + custom ML test harnesses
- Soda Data / Deequ data quality checks for ML features
- Airflow/Kubeflow test modules pipeline DAG testing
Model Inference / API Testing Tools
- Postman/Newman testing ML inference endpoints
- RestAssured API-based model validation
- Locust/JMeter load testing of inference throughput & latency
Observability & Monitoring Tools
- Prometheus + Grafana monitoring inference latency & performance
- Elastic Stack (ELK) logging of model behavior & anomalies
- OpenTelemetry tracing AI pipeline performance
Your Skills And Experience
Test automation using Selenium, Playwright, Cypress, or equivalent.
- Strong programming/scripting in Python, Java, or JavaScript.
- Framework design experience (POM, Hybrid, BDD, Data-Driven).
- Strong background in API testing.
- Experience with ML/AI testing frameworks (Evidently, Deepchecks, MLflow, Fiddler, GE).
- Understanding of ML workflows, model metrics, and data validation.
- Experience with cloud platforms and containers.
Soft Skills
- Strong analytical and debugging skills.
- Attention to detail and quality mindset.
- Ability to collaborate with engineering, AI, and product teams.
- Clear communication skills; ability to explain ML test findings.
How We'll Support You
- Training and development to help you excel in your career
- Coaching and support from experts in your team
- A culture of continuous learning to aid progression
- A range of flexible benefits that you can tailor to suit your needs
About Us And Our Teams
Please visit our company website for further information:
https://www.db.com/company/company.html
We strive for a culture in which we are empowered to excel together every day. This includes acting responsibly, thinking commercially, taking initiative and working collaboratively.
Together we share and celebrate the successes of our people. Together we are Deutsche Bank Group.
We welcome applications from all people and promote a positive, fair and inclusive work environment.
