Data Architect

Project Role : Data Architect

Project Role Description : Define the data requirements and structure for the application. Model and design the application data structure, storage and integration.

Must have skills : Data Architecture Principles

Good to have skills : NA

Minimum 7.5 Year(s) Of Experience Is Required

Educational Qualification : 15 years full time education

Summary:

We are seeking a highly skilled Senior Data Engineer / Technical Architect AWS to lead the design, development, and optimization of scalable data lake and ETL solutions in the AWS ecosystem.

Roles & Responsibilities:

Lead the design, architecture, and development of large-scale ETL and data lake solutions using AWS Glue, S3, and related services.

Develop, monitor, and optimize ETL pipelines for multi-source data ingestion, transformation, and integration into centralized data lakes.

Create and manage the AWS Glue Data Catalog, ensuring consistent metadata management, schema evolution, and version control.

Implement data governance, quality checks, and security policies using AWS Lake Formation, IAM, and encryption best practices.

Collaborate with cross-functional teams including data analysts, scientists, and business users to translate business needs into scalable technical solutions.

Design and manage CI/CD pipelines for AWS ETL code deployment using GitHub, CloudWatch, and Glue workflows.

Optimize PySpark scripts and ETL jobs for performance, cost efficiency, and scalability using partitioning, parallelism, and compression techniques.

Support downstream analytics and BI platforms by delivering clean, curated, and query-optimized datasets.

Provide technical leadership and mentorship to junior data engineers, ensuring adherence to best practices in design, coding, and documentation.

Ensure data lineage, monitoring, and alerting via AWS CloudWatch and Glue consoles.

Professional & Technical Skills:

AWS Data Engineering Services: Glue, S3, Athena, Redshift, Step Functions, Lambda, SNS, IAM, EventBridge, CloudWatch, Lake Formation, Iceberg, CLI.

Programming Languages: Python, PySpark, SQL.

ETL Development: Glue Jobs, Crawlers, Workflows, and Triggers for batch and near-real-time data processing.

Data Lake & Catalog Management: Schema evolution, metadata consistency, partitioning strategies, and cost optimization.

Data Quality & Governance: Validation frameworks, data profiling, lineage, and compliance with enterprise standards.

Integration: Redshift, Athena, QuickSight, and other downstream analytics/BI tools.

Automation & CI/CD: GitHub, Glue workflows, Step Functions, and CloudWatch for orchestration and deployment.

Big Data & IoT: Design and implementation of scalable big data and IoT-based data ingestion pipelines.

Databases: Relational and NoSQL databases, data modeling, and query optimization.