We are seeking experienced
ETL Data Test Automation Engineers with Databricks expertise to validate complex cloud data pipelines. Work
onsite to ensure data quality, integrity, and performance across ETL/ELT processes using Python automation, PySpark testing, and advanced SQL validations.
Key Responsibilities
- Design/execute ETL/ELT test automation frameworks for cloud data pipelines
- Validate Databricks pipelines (Notebooks, Jobs, Delta Lake, Unity Catalog)
- Write Python test automation using PyTest/UnitTest + Pandas/PySpark
- Perform complex SQL validations (joins, window functions, aggregations)
- Test data lineage, metadata management, SCD transformations
- Automate API/CLI testing for data platform components
- Collaborate with data engineers for test-driven development
Mandatory Technical Skills
- ETL Fundamentals: Data modeling (Star/Snowflake), SCDs, lineage
- Databricks: Notebooks, Jobs, Delta Lake, Unity Catalog, SQL Warehouse
- Python: PyTest/UnitTest, Pandas, PySpark for test data manipulation
- SQL: Complex joins, window functions, performance validation
- Cloud Testing: Data/ETL testing in AWS/Azure/GCP environments
- Experience: 6-8 years QA (3+ years data testing)
Skills: testing,sql,python,automation,cloud,etl,data
