Senior Data Engineer

We are seeking a highly skilled Senior Data Engineer to support the migration of SAS-based data platforms, ETL processes, reporting datasets, and analytical workloads to Google Cloud Platform (GCP) using DBT. The role involves designing, developing, optimizing, and supporting cloud-native data pipelines while ensuring data quality, scalability, security, and compliance.

The ideal candidate should possess strong expertise in SQL, Python, PySpark, DBT, BigQuery, and data warehousing concepts, with experience in large-scale data migration and modernization initiatives.

Key Responsibilities

SAS Migration & Modernization

• Analyze existing SAS datasets, ETL jobs, PROC SQL code, and SAS macros.
• Convert SAS transformation logic into DBT models and cloud-native data pipelines.
• Support migration of historical and incremental data from SAS platforms to GCP.
• Participate in migration assessments, code conversion, and reconciliation activities.

Data Engineering & Development

• Design and develop scalable ELT pipelines using DBT and BigQuery.
• Build and optimize data ingestion, transformation, and aggregation processes.
• Develop reusable and modular DBT models following best practices.
• Implement performance tuning for SQL and BigQuery workloads.
• Create and maintain data lineage and metadata documentation.

Data Quality & Reconciliation

• Design and implement data quality validation frameworks.
• Perform source-to-target reconciliation between SAS and GCP platforms.
• Investigate and resolve data inconsistencies and migration issues.
• Support automated testing and monitoring of data pipelines.

Cloud Platform Development

• Build and manage solutions using:
• BigQuery
• Cloud Storage
• Dataproc
• Cloud Composer (Airflow)
• Dataflow
• Support data security, access controls, and governance requirements.

Collaboration & Delivery

• Work closely with Data Architects, Business Analysts, Data Modelers, and QA teams.
• Participate in design reviews and sprint planning sessions.
• Estimate effort and provide technical input during project planning.
• Mentor junior engineers and enforce engineering best practices.

Required Technical Skills

Core Technologies

• DBT (Data Build Tool)
• Google BigQuery
• SQL
• Python
• PySpark
• Git

GCP Services

• BigQuery
• Cloud Storage
• Dataproc
• Cloud Composer (Airflow)
• Dataflow
• IAM

SAS Technologies

• SAS Base
• SAS Enterprise Guide
• PROC SQL
• SAS Macros
• SAS Data Sets

Data Engineering

• ETL / ELT Development
• Data Warehousing
• Data Pipeline Design
• Performance Optimization
• Data Validation & Reconciliation

Data Modeling

• Star Schema
• Snowflake Schema
• Dimensional Modeling
• Source-to-Target Mapping

Preferred Skills

• Experience with Banking or Financial Services data.
• Understanding of regulatory reporting datasets.
• Experience with Data Vault methodology.
• Knowledge of data governance and metadata management.
• Familiarity with CI/CD pipelines and DevOps practices.
• Exposure to Agile delivery models.

Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related field.
• 8-12 years of experience in Data Engineering.
• Minimum 3+ years of experience on GCP and BigQuery.
• Minimum 2+ years of hands-on experience with DBT.
• Experience in large-scale data migration or modernization programs.

Key Deliverables

• SAS-to-DBT conversion specifications.
• DBT transformation models.
• BigQuery data pipelines and tables.
• Source-to-target mappings.
• Data reconciliation reports.
• Data quality validation frameworks.
• Technical and operational documentation.