Search by job, company or skills

EisnerAmper

Senior Developer, AI Engineering

EisnerAmper
Bengaluru, India
3-6 Years
Save
  • Posted 9 hours ago
  • Be among the first 10 applicants
Early Applicant

Job Description

Job Description

A QA Engineer for AI Initiatives is responsible for ensuring the quality, reliability, fairness, and performance of AI/ML-powered products and systems. Unlike traditional QA, this role requires deep understanding of non-deterministic model behavior, data quality, and AI-specific failure modes such as hallucinations, bias, and model drift.

Key Responsibilities

  • Design and execute test strategies specifically for AI/ML models, LLM-based applications, and data pipelines
  • Develop automated test frameworks for model validation, regression testing, and performance benchmarking
  • Evaluate model outputs for accuracy, consistency, relevance, hallucination, and bias across diverse inputs
  • Test RAG (Retrieval-Augmented Generation) pipelines, chatbots, recommendation systems, and other AI-driven features
  • Collaborate with data scientists and ML engineers to define acceptance criteria and quality thresholds
  • Build and maintain evaluation datasets, ground truth sets, and adversarial test cases
  • Monitor models in production for drift, degradation, and anomalous behavior
  • Validate data quality, data pipelines, and feature stores that feed AI systems
  • Document defects, edge cases, and failure patterns specific to AI behavior
  • Ensure AI systems meet ethical, fairness, and compliance standards (bias audits, explainability checks)

Required Skills & Qualifications

  • Bachelor's or Master's degree in Computer Science, Engineering, or a related field
  • 3–6 years of QA experience, with at least 1–2 years in AI/ML quality assurance
  • Strong proficiency in Python for test automation and data analysis
  • Familiarity with LLM evaluation frameworks (e.g., RAGAS, DeepEval, Promptfoo, LangSmith)
  • Hands-on experience with testing tools: Pytest, Selenium, Postman, or similar
  • Understanding of ML lifecycle — training, validation, deployment, and monitoring
  • Knowledge of data quality tools and pipeline testing (Great Expectations, dbt tests)

Nice to Have

  • Experience with prompt engineering and red-teaming LLMs
  • Familiarity with MLOps platforms (MLflow, SageMaker, Vertex AI)
  • Knowledge of vector databases and embedding quality evaluation
  • Understanding of AI safety, responsible AI principles, and fairness frameworks
  • Experience with A/B testing and shadow deployment strategies

Soft Skills

  • Analytical and inquisitive mindset — comfortable challenging model outputs
  • Ability to think like both a user and an adversary (red-team thinking)
  • Strong documentation and communication skills
  • Collaborative approach with data science, engineering, and product teams
  • High attention to detail with a quality-first attitude

Preferred Location:

Bangalore

More Info

Job Type:
Permanent Job
Industry:
Other
Function:
Quality Assurance
Employment Type:
Full time

About Company

EisnerAmper

Job ID: 150899005

Jobs by Skill - IT
Jobs by Skill - Non IT
International Jobs
Last Updated: 07-07-2026 09:37:47 AM
Homejobs in Bengaluru / BangaloreSenior Developer, AI Engineering

Similar Jobs

Senior Developer - Python Backend

Arctic Wolf
Bengaluru, India

Skills:

Ml, Dynamodb, Lambda, RDS, AWS, Python, Kubernetes, Docker, Terraform, Jenkins, Git, EKS, Go, Ai

AI Lead/Senior Developer

Sopra Steria
S
Bengaluru, India

Skills:

Tensorflow, Pytorch, MongoDB, LangChain, Enterprise cloud infrastructure, Machine Learning and Deep Learning, Prompt engineering, NLP RAG architectures, CloudSQL, Vector databases, Scikit-learn, Agentic AI solutioning and Architecture, Production AI platform management, Pinecone, AI model deployment and inference optimization, AI ML solution architecture and deployment, Scalable distributed systems, Proficiency in Python, Google ADK, pgVector

Java Full Stack Senior Developer with React

Qualitest
Bengaluru, India

Skills:

Java, Github, Bdd, Springboot, Junit, Microservices, React Js, Jenkins, Git, Tdd, Docker, Plsql, Postman, Oracle, Azure, Jest, AWS, Playwright, K8S, Github Copilot, ReadyApi, microfrontend, oci

R&D Senior Developer GMS - IDC

Hitachi Energy
Bengaluru, India

Skills:

contingency analysis , state estimation , load flow , operation optimization , Network Analysis, Machine Learning, Java, Fortran, Windows, MATLAB, Linux, Python, AGC, Economic Dispatch, AI packages, Operator Training Simulator, Power system modeling, Simulation analysis, Numerical analysis methodology, Power Systems Simulators, Optimization toolkits, Generation Control

Databrick Senior Developer

Tata Consultancy Services
T
Bengaluru, India

Skills:

time travel , Data Migration, S3, AWS, Azure, lake-house patterns, data modelling ELT, Databrick cloud data warehouse, ADLS, resource monitors, RDBMS to Databrick cloud data warehouse, RBAC controls, query performance tuning, Databrick SQL, Zero copy clone, Big Data model techniques using Python, NoSQL data stores, virtual warehouse sizing, data sharing events, DWH and ETL concepts, data security and data access controls, Databrick utilities, stored Procedures