Lead and manage large-scale
Data Engineering and Data Modernization projects.Hands-on experience in managing Data projects end-to-end — effort estimation, scoping, project plan, timelines, team allocation, stakeholder management
Drive end-to-end delivery of
Data Lake build and migration initiatives.Hands-on experience with modern Data technologies like PySpark, SQL, CML, Python on any cloud; preferred GCP
Lead
PySpark migration and optimization projects, ensuring performance and scalability. Transform business requirements into Data solutions, manage risks, issues and dependencies
Design and implement modern data architectures, including
Data Lakes, Data Warehouses, and Lakehouse solutions.
Collaborate with business stakeholders, architects, and engineering teams to define data strategies and roadmaps.
Provide technical leadership and mentorship to Data Engineers and Developers.
Ensure best practices around:
- Data governance
- Data quality
- Security and compliance
- Performance optimization
Lead data platform modernization initiatives across cloud environments.
Review solution designs, architecture documents, and implementation approaches.
Manage project planning, resource allocation, risks, and delivery timelines.
Drive Agile delivery and ensure successful project execution.
Requirements
- Hands-on experience in managing Data projects end-to-end — effort estimation, scoping, project plan, timelines, team allocation, stakeholder management
- Hands-on experience with modern Data technologies like PySpark, SQL, CML, Python on any cloud; preferred GCP
- Transform business requirements into Data solutions, manage risks, issues and dependencies
- Proven experience in delivering:
- Data Lake implementation projects
- Data Lake migration programs
- PySpark migration projects
- Large-scale data transformation initiatives
Technical Skills
- Strong expertise in:
- Python
- PySpark
- Spark SQL
- SQL
- ETL/ELT frameworks
- Experience with:
- Hadoop ecosystem
- Data Lakes and Lakehouse architectures
- Distributed data processing frameworks
- Strong understanding of:
- Data modeling
- Data integration patterns
- Batch and real-time processing
- Experience with cloud platforms such as:
- Hands-on experience with:
- Data migration strategies
- Performance tuning and optimisation
- CI/CD and DevOps practices for data platforms
Preferred Skills
- Experience with:
- Databricks
- Delta Lake
- Apache Airflow
- Kafka
- Snowflake
- Kubernetes and Docker
- Experience in Banking, Financial Services, or other large enterprise environments
- Exposure to data governance and data quality frameworks
