Senior Developer, AI Engineering

3-6 Years

This job is no longer accepting applications

Job Description

Job Description

A QA Engineer for AI Initiatives is responsible for ensuring the quality, reliability, fairness, and performance of AI/ML-powered products and systems. Unlike traditional QA, this role requires deep understanding of non-deterministic model behavior, data quality, and AI-specific failure modes such as hallucinations, bias, and model drift.

Key Responsibilities

Design and execute test strategies specifically for AI/ML models, LLM-based applications, and data pipelines
Develop automated test frameworks for model validation, regression testing, and performance benchmarking
Evaluate model outputs for accuracy, consistency, relevance, hallucination, and bias across diverse inputs
Test RAG (Retrieval-Augmented Generation) pipelines, chatbots, recommendation systems, and other AI-driven features
Collaborate with data scientists and ML engineers to define acceptance criteria and quality thresholds
Build and maintain evaluation datasets, ground truth sets, and adversarial test cases
Monitor models in production for drift, degradation, and anomalous behavior
Validate data quality, data pipelines, and feature stores that feed AI systems
Document defects, edge cases, and failure patterns specific to AI behavior
Ensure AI systems meet ethical, fairness, and compliance standards (bias audits, explainability checks)

Required Skills & Qualifications

Bachelor's or Master's degree in Computer Science, Engineering, or a related field
3–6 years of QA experience, with at least 1–2 years in AI/ML quality assurance
Strong proficiency in Python for test automation and data analysis
Familiarity with LLM evaluation frameworks (e.g., RAGAS, DeepEval, Promptfoo, LangSmith)
Hands-on experience with testing tools: Pytest, Selenium, Postman, or similar
Understanding of ML lifecycle — training, validation, deployment, and monitoring
Knowledge of data quality tools and pipeline testing (Great Expectations, dbt tests)

Nice to Have

Experience with prompt engineering and red-teaming LLMs
Familiarity with MLOps platforms (MLflow, SageMaker, Vertex AI)
Knowledge of vector databases and embedding quality evaluation
Understanding of AI safety, responsible AI principles, and fairness frameworks
Experience with A/B testing and shadow deployment strategies

Soft Skills

Analytical and inquisitive mindset — comfortable challenging model outputs
Ability to think like both a user and an adversary (red-team thinking)
Strong documentation and communication skills
Collaborative approach with data science, engineering, and product teams
High attention to detail with a quality-first attitude

Preferred Location:

Bangalore

More Info

Job Type:

Industry:

Function:

Employment Type:

About Company

EisnerAmper

Job ID: 150899005

Jobs by Skill - IT

Jobs by Skill - Non IT

International Jobs

Last Updated: 15-07-2026 06:57:09 AM

Homejobs in Bengaluru / BangaloreSenior Developer, AI Engineering

Similar Jobs

Sr. Data Scientist, AI Engineering

trinity life sciences

5-8 yrs

Bengaluru, India

Skills:

MLops, Gcp, Neo4j, PostgreSQL, AWS, Python, Azure, Rdf, Apis, LangChain, LlamaIndex, cloud platforms, Weaviate, pgvector, GenAIOps, Pinecone, LLM-driven applications, LangGraph, RAG pipelines, architecture standards, Milvus, knowledge graph systems, agentic AI systems, vector databases, knowledge-grounding frameworks, GenAI microservices, Chroma, prompt engineering, Transformers

Senior Developer AI Platform

Bnp Paribas Bank

5-7 yrs

Bengaluru, India

Skills:

restful architecture , API security, PostgreSQL, Multithreading, React, Concurrency, Python, Java, Vue, Data Structures, Object Oriented Programming, Sql, Algorithms, Parallel Processing, Svelte, Async Programming, LangGraph, Message Oriented Middleware, CrewAI, LangChain, API lifecycle management, multiprocessing, LLM orchestration frameworks, enterprise-scale AI platforms, FastMCP, Google ADK

Airbag - Senior Developer

Bosch

5-10 yrs

Bengaluru, India

Skills:

lauterbach , targetlink , canalyser , Rtos, Embedded C, Capl, Spi, Rhapsody, MATLAB, Shell, Python, VB, Can, Git, Flexray, Perl, CANDELA studio, Canoe, ALM RTC, Jenkins CI, CANDiva, function generator, Oscilloscope, Uds, KWP, Simulink, make, Logiscope, CANstress

AI Engineering

ntt data north america

5-7 yrs

Bengaluru, India

Skills:

PostgreSQL, Spring Boot, Automated Testing, Microservices, React, Typescript, Javascript, Docker, Flask, Python, Java, Sql, Pandas, FastAPI, Rest Apis, Kubernetes, embeddings, vector databases, CI CD, LLM APIs, cloud-native platforms, observability, RAG, prompt orchestration

Senior Developer ( Cloud )

Arctic Wolf

5-7 yrs

Bengaluru, India

Skills:

Distributed Systems, Kafka, Windows, Nosql, React, Docker, Terraform, Sonarqube, Macos, Python, AWS, Aws Services, Cloudformation, Sql, Rabbitmq, Jenkins, Gcp, Linux, Ansible, Arm, Rest Apis, Azure, Kubernetes, Go, Intel, Release Automation, GitHub Actions, Microservices Architectures, make, Bazel