Experience: 2.00 + years
Salary: Confidential (based on experience)
Expected Notice Period: 30 Days
Shift: (GMT+05:30) Asia/Kolkata (IST)
Opportunity Type: Hybrid ()
Placement Type: Full Time Permanent position(Payroll and Compliance to be managed by: Tailored AI)
(*Note: This is a requirement for one of Uplers client - Tailored AI)
What do you need for this opportunity
Must have skills required:
OTEL, DeepEval, RAGAS
Tailored AI is Looking for:
QA Engineer – AI Applications
Quality Assurance | 2–3 Years Experience | Hybrid / Remote
We're building AI systems that change how industries work. As a QA Engineer, you'll own quality across product tracks — working closely with engineers and clients to make sure what we ship actually works.
What You'll Do
- Own quality for an AI product track — from requirements to release
- Test LLM features, RAG pipelines, and multi-agent workflows
- Build automated test suites to catch regressions in AI outputs
- Run adversarial tests — find failure modes before users do
- Work with engineers and clients to define what 'good looks like
- Plug quality gates into CI/CD pipelines
- Track bugs to closure and flag risks clearly
What You'll Gain
- Hands-on experience testing AI systems across real industries
- Understanding of how LLMs and multi-agent systems break — and how to prevent it
- High ownership, fast feedback, and work that ships
- Clear path to Senior QA as we grow
Who You Are
- 2–3 years in QA or software testing, at least 1 year on AI/ML products
- Can write Python to automate tests and evaluate model outputs
- Have tested LLMs — prompts, outputs, hallucinations, edge cases
- Know REST API testing, pytest, and basic CI/CD
- Think in failure modes and communicate findings clearly
- Bonus: exposure to vector DBs, RAG, or AI bias testing
TECH STACK YOU'LL WORK WITH
Python
pytest
REST APIs
Postman
GitHub Actions
LLM APIs
LangChain
Vector DBs
AWS
Postgres
Redis
Good to have:
- DeepEval — LLM evaluation framework (hallucination, relevancy, toxicity testing)
- Ragas — evaluation framework specifically for RAG pipelines
SOME REAL PROBLEMS YOU MIGHT WORK ON
- Evaluating a multi-agent career coaching assistant for consistency and accuracy
- Red-teaming an LLM that generates employee performance reviews from HR data
- Testing a legal AI tool that surfaces case law and precedents for Indian lawyers
- Stress-testing a RAG pipeline as document volumes scale
INTERVIEW PROCESS
- Screening — Quick call with a Co-Founder to align on fit
- Live Testing Round — Given an AI feature, design and execute a test plan
- Technical Discussion — Walk us through your approach and how you'd automate it
How to apply for this opportunity
- Step 1: Click On Apply! And Register or Login on our portal.
- Step 2: Complete the Screening Form & Upload updated Resume
- Step 3: Increase your chances to get shortlisted & meet the client for the Interview!
About Uplers:
Our goal is to make hiring reliable, simple, and fast. Our role will be to help all our talents find and apply for relevant contractual onsite opportunities and progress in their career. We will support any grievances or challenges you may face during the engagement.
(Note: There are many more opportunities apart from this on the portal. Depending on the assessments you clear, you can apply for them as well).
So, if you are ready for a new challenge, a great work environment, and an opportunity to take your career to the next level, don't hesitate to apply today. We are waiting for you!
