Must-Have Skills:

Hands-on experience with Databricks and Apache Spark for building and validating scalable data pipelines
Strong expertise in AWS services including S3, Glue, Athena, Redshift, and Lake Formation
Proficient in Python, PySpark, and SQL for developing test automation and validation logic
Experience validating data from various file formats such as JSON, CSV, Parquet, and Avro
In-depth understanding of data integration workflows including batch and real-time (streaming) pipelines
Strong ability to define and automate data quality checks: schema validation, null checks, duplicates, thresholds, and transformation validation
Experience designing modular, reusable automation frameworks for large-scale data validation
Skilled in integrating tests with CI/CD tools like GitHub Actions, Jenkins, or Azure DevOps
Familiarity with orchestration tools such as Apache Airflow, Databricks Jobs, or AWS Step Functions
Hands-on experience with API testing using Postman, pytest, or custom automation scripts
Proven track record of leading and mentoring QA/test engineering teams
Ability to define and own test automation strategy and roadmap for data platforms
Strong collaboration skills to work with engineering, product, and data teams
Excellent communication skills for presenting test results, quality metrics, and project health to leadership
Contributions to internal quality dashboards or data observability systems
Awareness of metadata-driven testing approaches and lineage-based validations
Experience working with agile Testing methodologies such as Scaled Agile.
Familiarity with automated testing frameworks like Selenium, JUnit, TestNG, or PyTest.

Software Development Engineer III

Software Development Engineer III