Machine Learning QA

Job Description

Role Summary

We are looking for an experienced Machine Learning QA Engineer with strong Python programming skills and hands on experience in ML testing, data quality validation, and automation framework development.

This role involves building an end to end ML QA automation framework, validating ML models, and ensuring the reliability and accuracy of AI/ML systems.

Key Responsibilities

1. ML Testing & Validation

Develop test strategies for ML models

Validate ML outputs using metrics such as Accuracy, Recall, Precision, F1-score

Create automation tests for model training and inference.

Validate feature engineering, preprocessing, and data transformations and model results.

2. Python Automation & Framework Development

Design and implement a scalable Python-based ML QA framework using:

o PyTest

o Pandas / NumPy

Build reusable libraries for:

o Data validation

o Model performance

o Model evaluation

Integrate the framework with CI/CD tools (GitHub Actions, Jenkins, GitLab).

3. Data Quality Assurance

Validate input datasets for:

o Schema

o Missing values

o Data/feature/concept drift

o Fairness & bias test

Work with large CSV/Parquet datasets for ML pipelines.

4. MLOps & Deployment Validation

Validate CI/CD pipelines for ML deployments.

Perform model version comparison & baseline checks.

Validate model packaging (Docker-based) and reproducibility

Requirements

Required Skills (Must Have)

Python

Excellent Python programming (3+ years)

Strong in Pandas, NumPy, PyTest/unittest

ML/AI

Understanding of:

o Machine learning and AI

o Model metrics and evaluation

o Data preprocessing

o Feature engineering

Tools & Tech

VS Code

Git/GitHub

Linux & Shell scripting

Docker basics

API testing (Requests, Postman)

Data Skills

SQL queries

Data profiling

Working with big datasets

QA Skills

Strong QA fundamentals.

Ability to design test scenarios for ML models and data pipelines.

Hands on with automation testing using Python (PyTest/unittest).

Experience with data validation testing (schema, quality, consistency, drift).

Ability to perform performance and load testing for ML inference services.

Familiarity with CI/CD testing and integrating tests into pipelines.

Good debugging, log analysis, and issue triaging skills.

Preferred Skills (Short Version)

Experience in Machine learning testing.

Good Knowledge on Python language.

Basic knowledge of ML pipelines.

Understanding of anomaly detection / predictive maintenance concepts.

Familiarity with cloud ML platforms (AWS / Azure / GCP) and MLOps workflows.

Qualifications

Bachelor's/Master's degree in Computer Science, Data Science, AI/ML, or related field

35 years total experience in QA with at least 12 years in ML/AI testing