datasets, ensuring rigorous data quality and reconciliation across our ecosystem.

Responsibilities

Pipeline Engineering: Develop and maintain robust batch, streaming, CDC, and API-based data pipelines, historical snapshots, and SCD frameworks.
Data Transformation: Build transformations for EDW, data lakes, and semantic layers, including specialised payment harmonisation datasets.
Performance & Quality: Tune SQL queries and Spark jobs, implementing stringent reconciliation controls, error handling, and operational monitoring.
SDLC Delivery: Support SIT, UAT, defect resolution, release execution, and production verification in close collaboration with architecture and BI teams.

Requirements:

Experience: Minimum 6 years of hands-on data engineering experience, with independent ownership of development and testing deliverables.
Technical Stack: Advanced proficiency in SQL, Python, Spark/PySpark, Teradata, and Hadoop/Cloudera ecosystems.
Tools & Integration: Experience with Informatica/Talend, Hive, Impala, Kafka, Airflow, APIs, and Git-based version control.
Domain Exposure: Strong background in enterprise data warehousing, data lakes, and banking data operations (Payments, Risk, Finance, or AML preferred).
Core Competencies: Independent ownership of deliverables, analytical mindset, and a commitment to secure, high-quality engineering standards.

Backend Data Platform Engineer (SQL / Python /PySpark)