Data Tester (ETL + Data Testing)

Role Context

Ensure quality assurance for data platforms and ETL pipelines through black-box and white-box testing, with strong coverage across integration testing, performance testing, and ETL automation testing. The tester will create test plans, test cases, and scripts, and work closely with data engineers, architects, and BAs to validate transformation correctness, integrity, and operational readiness.

Key Responsibilities

Perform black-box testing (validating outputs against requirements without internal logic dependency).

Perform white-box testing (validating transformation logic, joins, filters, dependencies, and code paths).

Design and execute integration testing across multi-source ingestion, transformation layers, and downstream consumption.

Execute performance testing for ETL workloads: runtime benchmarking, volume testing, concurrency/load impact, and regression detection.

Prepare and maintain test artifacts: test strategy, test plan, test cases, test scripts, and traceability matrix.

Implement ETL automation testing for regression suites and repeatable validations (SQL harness, frameworks, CI-driven execution).

Validate data quality: completeness, uniqueness, consistency, referential integrity, business rule correctness.

Perform reconciliation testing: source-to-target counts/sums, delta validations, incremental loads, and backfill correctness.

Support release readiness: UAT support, defect triage, RCA collaboration, and sign-off.

Must Have

Strong SQL skills to validate transformations, aggregates, and reconciliation rules.

Proven experience testing ETL pipelines and data transformations (batch and/or near real-time).

Hands-on experience with:

Integration testing across data layers and pipelines

Performance testing for ETL jobs (volume, runtime, regression analysis)

Black-box and white-box testing methodologies

Experience creating test plans, test scripts, and executing structured test cycles.

Exposure to ETL testing automation approaches:

SQL-based automation harness, dbt tests, Great Expectations, or comparable framework-based validation.

Understanding of SCD/CDC, incremental loads, dependencies, and reprocessing/backfills.

Delivery/DevOps fundamentals: Git usage, CI/CD familiarity for automated test execution, environment promotions.

Non-technical: strong attention to detail, documentation discipline, stakeholder communication, ability to work with engineers to reproduce issues.

Good To Have

Python for automation scripting (pytest-based validation, framework extension).

Experience with orchestration tools (Airflow) and validating DAG behaviors (retries, SLAs, backfills).

Observability exposure: log analysis, monitoring dashboards, alert validation.

Knowledge of test data management strategies (synthetic datasets, masking, controlled fixtures).

Exposure to cloud data platforms and warehouses (AWS/Azure/GCP + Snowflake/BigQuery/Redshift etc.).

Familiarity with API testing if pipelines expose data services.

6) Python and Prompt engineering:

Prompt Engineer & Generative AI Specialist (25 Years Experience - can be less experience too)

Job Summary:

We are seeking a skilled Prompt Engineer & Generative AI Specialist to optimize interactions with advanced language models and generative systems. You will work closely with AI product managers, data scientists, and software developers to craft and refine prompts, ensuring that model outputs align with business requirements and user expectations.

Key Responsibilities:

Prompt Development & Optimization:

Design, test, and iterate on prompts for generative AI models to achieve accurate, relevant, and contextually appropriate outputs.

Continuously refine and improve prompt strategies based on performance metrics and user feedback.

Cross-Functional Collaboration:

Partner with AI product managers, data scientists, and software engineers to integrate prompt engineering best practices into the product lifecycle.

Translate business requirements into effective prompt frameworks and fine-tuning strategies.

Research & Innovation:

Stay up-to-date with the latest advancements in natural language processing (NLP) and generative AI methodologies.

Experiment with novel prompt engineering techniques and contribute to internal knowledge bases and best practice guidelines.

Quality & Ethical Oversight:

Ensure ethical and responsible usage of generative AI by minimizing biases and adhering to established guidelines.

Monitor and evaluate the performance of deployed prompts, making adjustments as needed to maintain high standards of quality and compliance.

Documentation & Reporting:

Document prompt engineering processes, experiments, and results to support knowledge sharing and continuous improvement.

Provide regular updates and technical reports to stakeholders on performance and enhancement initiatives.

Qualifications:

Bachelor's or Master's degree in Computer Science, Computational Linguistics, Data Science, or a related field.

25 years of experience in roles involving natural language processing, prompt engineering, or generative AI applications.

Strong familiarity with large language models (e.g., GPT, ChatGPT) and generative AI frameworks.

Proficiency in Python or other scripting languages for interacting with AI models.

Proven experience in designing, testing, and optimizing prompts for various applications.

Excellent analytical, problem-solving, and communication skills, with the ability to work effectively in cross-functional teams.

A strong commitment to ethical AI practices and a deep understanding of AI fairness and bias mitigation strategies.