Key Responsibilities:
Data Engineering Architecture & Development
- Lead the design, development, and implementation of data engineering solutions on AWS cloud.
- Architect and build scalable data pipelines using Spark, PySpark, and Python.
- Design and implement data lake and data warehouse solutions for analytics and reporting use cases.
Data Processing & Optimization
- Develop optimized SQL queries and transformations for large-scale data processing.
- Ensure high performance, scalability, and efficiency of data pipelines.
- Implement best practices for ETL development and data processing workflows.
Data Modeling & Warehousing
- Create and maintain dimensional and normalized data models.
- Support business intelligence and advanced analytics requirements.
- Ensure data consistency, integrity, and governance across systems.
Data Quality & Governance
- Implement data quality checks and validation frameworks.
- Ensure adherence to data governance and compliance standards.
- Monitor and improve data reliability and accuracy.
Leadership & Collaboration
- Provide technical leadership and mentorship to data engineering teams.
- Collaborate with data architects, BI teams, and business stakeholders.
- Translate business requirements into scalable technical solutions.
Architecture & Delivery
- Participate in architecture discussions, design reviews, and project planning.
- Drive best practices in coding standards, performance tuning, and optimization.
- Ensure timely delivery of data engineering solutions.
