Title: Big Data Test Engineer with Python/Shell Script Automation
Location: Pune
Requirements:
- Bachelor's or master's degree in computer science, Engineering, or a related field.
- 8+ years of experience in software testing and development, with a focus on data-intensive applications.
- Proven experience in testing data pipelines and ETL processes - Test planning, Test Environment planning, End to End testing, Performance testing.
- Solid programming skills in Python - proven automation effort to bring efficiency in the test cycles.
- Solid understanding of Data models and SQL.
- Must have experience with ETL (Extract, Transform, Load) processes and tools (Scheduling and Orchestration tools, ETL Design understanding)
- Good understanding of Big Data technologies like Hadoop, Spark, Hive, and Impala.
- Understanding of Data Warehouse methodologies, applications, and processes.
- Experience working in an Agile/Scrum environment, with a solid understanding of user stories, acceptance criteria, and sprint cycles.
Role:
- Support Development, and maintain automated test frameworks, tools, and test cases for Data Engineering and Data Warehouse applications.
- Collaborate with cross-functional teams, including software developers, data engineers, and data analysts, to ensure comprehensive testing coverage and adherence to quality standards.
- Conduct thorough testing of data pipelines, ETL processes, and data transformations using Big Data technologies.
- Apply your knowledge of Data Warehouse/Data Lake methodologies and best practices to validate the accuracy, completeness, and performance of our data storage and retrieval systems.
- Identify, document, and track software defects, working closely with the development team to ensure timely resolution.
- Participate in code reviews, design discussions, and quality assurance meetings to provide valuable insights and contribute to the overall improvement of our software products.
Nice to have:
- Experience with scripting languages like Bash or Shell.
- Experience working with large-scale datasets and distributed data processing frameworks (e.g., Hadoop, Spark).
- Familiarity with data integration tools like Apache NiFi is a plus.
- Excellent problem-solving and debugging skills, with a keen eye for detail.
- Strong communication and collaboration skills to work effectively in a team-oriented environment.
- Eagerness to learn and contribute to a growing team.
