Quality Assurance Engineer

About US:

Straive is a global leader in data analytics and AI operationalization, helping enterprises embed advanced AI and data capabilities into core business workflows to deliver measurable business outcomes and ROI. With a workforce of 20,000 professionals serving 350+ clients across 30+ markets, Straive combines technical scale with deep domain expertise. A key differentiator is its network of over 6,000 subject matter experts who specialize in managing and enriching complex, unstructured data enabling organizations to build AI systems grounded in accuracy, context, and business relevance.

The company continues to earn recognition from leading industry analysts and was recently named a Leader in AIM's 2026 Generative AI and Data Engineering PeMa Quadrants. Backed by EQT a purpose-driven global investment organization which was ranked among the world's leading private equity firms by PEI in 2025 Straive is positioned as a high-value alternative to traditional IT services providers, combining domain-led intelligence with AI execution at scale.

Role & responsibilities

•Design and execute comprehensive test strategies tailored for AI/ML models, LLM-based

applications, and data pipelines to ensure system accuracy and reliability.

•Develop and maintain automated test frameworks for model validation, regression testing,

and performance benchmarking using Pytest.

• Evaluate LLM and RAG outputs for structural correctness, factual consistency, relevance,

hallucination risks, and demographic bias.

•Build, update, and manage robust evaluation datasets, golden/ground truth sets, and

adversarial edge-case test suites.

•Monitor production models to track performance metrics, identify data/model drift, and

catch anomalous behaviors over time.

•Validate incoming data streams, data pipelines, and feature stores to confirm completeness

and accuracy of training/inference data.

•Collaborate directly with Data Scientists and Machine Learning Engineers to define explicit

acceptance criteria, quality thresholds, and prompt regression standards.

•Conduct ethical and compliance assessments, including bias audits and model explainability

checks.

•Document distinct defects, edge cases, and failure patterns specific to non-deterministic AI

behaviors.

Preferred profile

Experience: 3 to 6 years of core QA experience, with a mandatory 1 to 2 years specialized in AI/ML

quality assurance and LLM testing.

Education: Bachelor's or Master's degree in Computer Science, Engineering, or a closely related

technical field.

Technical Expertise: Strong hands-on coding in Python for data analysis and test automation.

Tools Knowledge: Proven familiarity with LLM evaluation frameworks (RAGAS, DeepEval,

Promptfoo, LangSmith) alongside traditional QA tools (Pytest, Selenium, Postman).

Data Skills: Working knowledge of data quality frameworks such as Great Expectations or dbt test

structures.

Domain Context: Solid understanding of the Machine Learning lifecycle (training, validation,

deployment, monitoring) and RAG system architectures.

Job Location - Bangalore or Mumbai

Work mode- Hybrid

Straive is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. All employment decisions are based on business needs, job requirements, and individual qualifications, without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, veteran, or disability status.