Job Description:

As a Data Engineer you will design and implement data ingestion canonical mapping validation lineage and data processing capabilities for an enterprise platform supporting regulated workflows
You will work with architects backend engineers AI engineers QA and domain specialists to ensure data is reliable traceable governed and reusable across modules

Key Responsibilities:

Design and build data ingestion pipelines for files APIs SFTP sources databases and enterprise data extracts
Implement canonical data models source to target mapping transformation rules validation checks rejected record handling and lineage capture
Develop data quality checks reconciliation routines control result tables exception datasets and metadata structures for auditable data processing
Work with PostgreSQL SQL object storage data lake patterns ETL ELT tools and batch processing frameworks as needed
Partner with AI engineers to provide clean contextual and grounded data for AI summaries narratives retrieval and explanations
Create data dictionaries mapping specifications interface contracts test datasets and data quality documentation
Support performance tuning partitioning indexing data retention archival and privacy aware data handling
Work with QA teams to defi

Technical Requirements:

Minimum 7 years of experience in data engineering ETL ELT development data warehousing data platform engineering or related roles
Strong SQL skills and hands on experience with relational databases such as PostgreSQL SQL Server Oracle or MySQL
Experience designing ingestion pipelines transformations validation rules data quality checks and source to target mapping specifications
Understanding of data modeling canonical models metadata lineage audit fields reconciliation and data quality concepts
Experience with Python Spark Databricks Azure Data Factory dbt Airflow or equivalent data engineering tools is preferred
Ability to work with structured semi structured and file based data formats such as CSV JSON Excel XML and Parquet
Strong documentation discipline and ability to collaborate with functional SMEs and engineers on data requirements

Preferred Skills:

Technology->Analytics - Packages->Python - Big Data,Technology->Data Engineering->Databricks,Technology->Data Services Testing->Data Warehouse Testing,Technology->ETL & Data Quality->ETL - Others

Data Engineer - RegTech