Experience: 27 years
Location: Mumbai / Gurgaon / Bangalore (flexible)
Department: Testing & Quality Engineering
Key words marked in red.
About the role
We're hiring a hands-on Test Automation Engineer who understands modern automated testing across UI, API/back-end, performance, security and critically - testing of LLM-driven/agentic automations and conversational agents. This person will build, run and own automated test suites end-to-end, validate agent behaviors, and work with product & engineering teams to maintain high quality in increasingly agent-enabled systems.
Key Responsibilities
- Design, develop and maintain automated test suites for UI, API, backend systems and conversational agents (LLM-based agents).
- Create and own JMeter scripts for performance, load, stress and endurance testing; analyse results and recommend tuning.
- Test and validate agentic workflows: multi-step agent conversations, orchestration between agents, failure modes, recovery and escalation flows.
- Validate conversational AI / chatbots using frameworks/tools such as Botium, Dialogflow, TestMyBot(or equivalents): intent recognition, slot-filling, conversation flows, fallback handling and NLU accuracy.
- Perform UI/UX validation: visual consistency, navigation flows, accessibility checks, and cross-browser/device behaviour (use BrowserStack, Katalon or similar).
- Build and maintain regression suites (Selenium / Katalon / Playwright)
- Develop automated API test suites (Postman / REST-assured / custom scripts) and validate API SLAs, response correctness, and performance.
- Integrate tests with CI/CD (Jenkins/GitLab/Azure DevOps) and produce actionable dashboards and test reports.
- Triage defects, collaborate with devs/product on fixes, and re-run automated pipelines to validate fixes.
- Drive continuous improvement: enhance automation coverage, introduce best practices for testing agentic systems, and document test strategy.
Required skills & experience
- 27 years of experience in software testing with strong automation exposure.
- Proven hands-on experience with Apache JMeter for performance/load testing.
- Solid experience building functional automation using Selenium, Katalon or equivalent frameworks
- Experience testing conversational agents / chatbots / voice bots using tools like Botium, Dialogflow, TestMyBot or similar.
- Familiarity with LLM/agent testing concepts: prompt/capability testing, hallucination detection, safety checks, conversation drift, and chain-of-thought evaluation.
- Comfortable writing code/scripts in Java / Python / JavaScript / Groovy for automation tasks.
- Experience integrating tests with CI/CD pipelines and generating meaningful dashboards/reports.
