ETL DB Testing Hadoop

Responsibilities

• Design and implement ETL processes for data extraction, transformation, and loading in Hadoop environments.
• Develop and execute test cases for validating ETL processes and data accuracy.
• Perform data quality checks and ensure data integrity throughout the ETL pipeline.
• Collaborate with data engineers and developers to troubleshoot and resolve issues related to data processing.
• Document test plans, test cases, and test results for ETL processes.
• Participate in code reviews and provide feedback on ETL scripts and processes.

Skills and Qualifications

• Strong experience with Hadoop ecosystem components such as HDFS, Hive, Pig, and Spark.
• Proficiency in SQL and experience with data warehousing concepts.
• Knowledge of ETL tools like Talend, Informatica, or Apache Nifi.
• Experience in writing and executing test cases for ETL processes.
• Familiarity with programming languages such as Java or Python.
• Understanding of data modeling and data governance best practices.
• Ability to work in a fast-paced team environment and manage multiple priorities.