Job Title: AI | QA Engineer
Location: Bangalore, India (On-site)
Department: AI & Automation
About The Company
Opkey is the leading Cloud Application Lifecycle Management (CALM) platform for Oracle, Workday, Salesforce, Coupa, and more. It cuts the costs and risks that drag down implementations and ongoing change, helping you go live on time, get more from your cloud app investments, and reach AI readiness faster. Opkey's 20+ AI agents manage all five phases of the cloud application lifecycle—Define, Design, Configure, Test and Train.
Whether it's a new implementation, a platform update, or business‑as‑usual change, Opkey handles it all: updates validated in hours, self‑healing tests, end‑to‑end integrations assured, configurations synced, and training updated in real time—all delivered in a single unified platform instead of a patchwork of disconnected tools.
Powered by Argus, a domain‑specific AI model trained on decades of expertise and terabytes of enterprise application data, Opkey automates configuration, testing, change impact analysis, and training across these applications—cutting manual effort by 80%, enabling 30% faster go‑lives, and slashing downtime risk by 92%
About The Role
We are looking for a skilled AI QA Engineer to join our ERP automation team at Opkey. You will work closely with AI developers to test, validate, and ensure the quality of AI-powered agents built for enterprise ERP workflows.
Key Responsibilities
- Design, develop, and execute test strategies for AI agents used in ERP automation pipelines
- Validate the behavior, accuracy, and reliability of AI agent outputs across various ERP scenarios
- Collaborate with AI developers to understand agent architecture and identify edge cases
- Build and maintain automated test frameworks tailored for LLM-based and agentic systems
- Perform functional, regression, integration, and performance testing on AI-driven workflows
- Monitor AI agent performance post-deployment and flag anomalies or drift in behavior
- Write detailed test plans, bug reports, and QA documentation specific to AI systems
- Conduct prompt testing and evaluate AI responses for correctness, consistency, and safety
- Build and maintain internal QA dashboards and reporting tools for real-time test visibility
- Participate in Agile/Scrum ceremonies including sprint planning, standups, retrospectives, and backlog grooming as the QA representative
- Conduct thorough Root Cause Analysis (RCA) on defects, test failures, and AI agent misbehaviors and present findings to the team
- Follow and contribute to SDLC and STLC processes ensuring quality gates are met at every stage of development
Required Skills & Experience
- 2–4 years of hands-on experience as a QA Engineer in software or automation products
- Strong understanding of SDLC and STLC methodologies with experience applying them in fast-paced product environments
- Hands-on experience working in Agile/Scrum teams with familiarity in tools like Jira or Azure DevOps
- Proficiency in Python for writing test scripts, automation utilities, and data validators
- Experience building lightweight internal tools or test dashboards using Streamlit
- Solid understanding of AI agents, LLMs, RAG pipelines, and agentic frameworks (LangChain, AutoGen, CrewAI, etc.)
- Experience testing API-based systems, microservices, and complex backend workflows
- Proficiency with test automation tools such as Selenium, Pytest, Postman, or similar
- Proven ability to perform Root Cause Analysis on complex bugs and systemic failures with clear documentation
- Familiarity with ERP platforms (SAP, Oracle, Salesforce, etc.) is a strong plus
- Knowledge of prompt engineering and how to test AI model behavior systematically
- Experience with CI/CD pipelines and integrating automated tests into DevOps workflows
AI Certifications (Required / Preferred)
- Certification in AI/ML fundamentals — Google, Microsoft (Azure AI), AWS, or IBM preferred
- Certification in Generative AI or LLM-related technologies (e.g., DeepLearning.AI, Coursera, Udacity)
- Any recognized certification in AI Agent development or Agentic frameworks is a strong plus
Skills: llm,pytest,ai agents,selenium,postman,rag pipelines
