ETL Python

P1, C3, STS

• Design, build data cleansing and imputation, map to a standard data model, transform to satisfy business rules and statistical computations, and validate data content
• Develop, modify, and maintain Python and Unix Scripts, and complex SQL
• Performance tuning of the existing code and avoid bottlenecks and improve performance
• Build an end-to-end data flow from sources to entirely curated and enhanced data sets
• Develop automated Python jobs for ingesting data from various source systems
• Provide technical expertise in areas of architecture, design, and implementation
• Work with team members to create useful reports and dashboards that provide insight, improve/automate processes, or otherwise add value to the team
• Write SQL queries for data validation
• Design, develop and maintain ETL processes to extract, transform and load data from various sources into the data warehouse
• Collaborate with data architects, analysts and other stakeholders to understand data requirements and ensure quality
• Optimize and tune ETL processes for performance and scalability
• Develop and maintain documentation for ETL processes, data flows, and data mappings
• Monitor and troubleshoot ETL processes to ensure data accuracy and availability
• Implement data validation and error handling mechanisms
• Work with large data sets and ensure data integrity and consistency

Skills

• Python
• ETL Tools like Informatica, Talend, SSIS or similar
• SQL, MySQL
• Expertise in Oracle, SQL Server and Teradata
• DevOps, GIT Lab
• Experience in AWS Glue or Azure Data Factory

Qualification

• Design, build data cleansing and imputation, map to a standard data model, transform to satisfy business rules and statistical computations, and validate data content
• Develop, modify, and maintain Python and Unix Scripts, and complex SQL
• Performance tuning of the existing code and avoid bottlenecks and improve performance
• Build an end-to-end data flow from sources to entirely curated and enhanced data sets
• Develop automated Python jobs for ingesting data from various source systems
• Provide technical expertise in areas of architecture, design, and implementation
• Work with team members to create useful reports and dashboards that provide insight, improve/automate processes, or otherwise add value to the team
• Write SQL queries for data validation
• Design, develop and maintain ETL processes to extract, transform and load data from various sources into the data warehouse
• Collaborate with data architects, analysts and other stakeholders to understand data requirements and ensure quality
• Optimize and tune ETL processes for performance and scalability
• Develop and maintain documentation for ETL processes, data flows, and data mappings
• Monitor and troubleshoot ETL processes to ensure data accuracy and availability
• Implement data validation and error handling mechanisms
• Work with large data sets and ensure data integrity and consistency