Job Title: SDET PySpark/Python Data Engineering (Azure Cloud)
Location: Indore | Pune | Bangalore | Chennai
Experience Required: 5+ years
Role Overview:
Seeking a skilled Software Development Engineer in Test (SDET) with expertise in PySpark and Python for data engineering and data pipeline validation on Azure Cloud. You will play a critical role in building automation frameworks, developing and executing test cases to ensure the quality and reliability of large-scale data solutions.
Key Responsibilities:
- Develop, maintain, and execute automated test scripts for data pipelines and engineering workflows using Python and PySpark.
- Validate data transformations, data quality, and pipeline performance on Azure Cloud environments (preferably using Databricks).
- Collaborate with data engineers and developers to understand data workflows and test scenarios.
- Build and optimize test frameworks that integrate with CI/CD pipelines.
- Perform thorough regression, performance, and API testing for data applications.
- Report and track defects, and work closely with engineering teams for resolution.
Skills & Qualifications:
- Strong programming skills in Python and PySpark.
- Experience in automated testing for data pipelines and big data platforms.
- Working knowledge of Azure Cloud services, including Azure Data Factory and Databricks.
- Familiarity with CI/CD tools and test automation frameworks.
- Understanding of data validation, ETL testing, and SQL query optimization.
- Excellent problem-solving and communication skills.
Preferred:
- Hands-on experience with Databricks and related Azure data technologies.
- Background in software development and test engineering for data-intensive applications.
