Machine Learning Engineer (Data & Model Validation) - Remote position

Job Description

We are looking for an ML Engineer focused on Data and Model Validation to ensure the correctness, reliability, and production readiness of machine learning systems. This role owns data quality, model evaluation, drift detection, and release validation for ML pipelines deployed at scale.

This is not a traditional QA role it is an engineering role with ownership of ML quality.

Experience - 2+ years

Key Responsibilities

Own data validation and quality checks for ML pipelines (training & inference)

Define and implement baseline expectations on datasets and features

Validate ML models using metrics such as F1, precision, recall, AUC

Perform regression testing across model retraining cycles

Detect and respond to data drift, feature drift, and prediction drift

Implement automated quality gates that block or allow model releases

Integrate validation checks into CI/CD and MLOps pipelines

Validate batch and real-time inference systems

Define model release readiness criteria

Document validation logic and monitoring thresholds

Mandatory Skills

Hands-on experience with Great Expectations or Deequ

Strong Python skills (Pandas, NumPy, PyTest)

Strong understanding of ML fundamentals

Experience validating ML models in production

Experience with ML pipelines

Familiarity with CI/CD pipelines

Good to Have

Experience with Scikit-learn, TensorFlow, or PyTorch

Experience with MLflow, Azure ML, SageMaker, or Vertex AI

Experience with LLM / GenAI systems

Cloud experience (AWS / Azure / GCP)

What This Role Is Not

Not a UI testing role

Not manual QA

Not Selenium-based testing

Hiring Note

Candidates must demonstrate hands-on experience validating ML data and models using Great Expectations or equivalent tools.