Senior Executive, Data Integration (pySpark, Nifi, Hadoop)

The Role

• Implement ETL systems that are operationally stable, efficient and automated. This include technical solutions that are scalable, aligned with the enterprise architecture and can adapt to changes in business.
• Work closely with internal/external teams devise requirements of data integrations, specific to Data Warehouse/Data Marts implementations.

Responsibilities of the Role

• Review business and technical requirements and ensure the data integration platform meets requirements.
• Apply industry best practices for ETL design and development.
• Produce written deliverables for technical design, system testing and implementation activities.
• Conduct System Testing - execute job flows, investigate system defects, resolve defects and document results.
• Work with DBAs, application specialists and technical services to tune performance of the ETL system to meet performance standards and SLAs.
• Assist in the development, documentation and application of best practices and procedures

Competencies

• Strong knowledge in SQL writing skills.
• Familiarity with ETL tools like pySpark, NiFi and Hadoop are preferred.
• Understanding of data integration best practices, leading industry applications and features such as master data management, entity resolution, data quality assessment, metadata management, etc.
• Exposure in data warehouse architecture, data analysis/data profiling of source systems
• Able to function well in a fast-paced and adaptive environment.
• Financial domain experience is a plus
• Ability to work independently and communicate across multiple levels (Product owners, Executive sponsors, Team members)
• Work experience in Agile methodology is a plus

Qualifications

• Minimum Bachelor's Degree in computer science/Information Technology or equivalent
• Candidates should have more than 5+ years total work experience, with experience programming ETL programs using Informatica, Nifi, pySpark and Hadoop
• 4+ years experience performing data analysis and profiling, and designing ETL systems/programs.