Role Objective
We are seeking a highly skilled Lead Data Engineer / Tech Lead - Data Engineering to join our dynamic team. In this role, you will not only design and develop our cloud-based data infrastructure but also lead a team of Data Engineers to deliver robust and scalable data solutions for our BFSI customers. As a Tech Lead, you will work at the intersection of cloud technologies, data engineering, and the BFSI domain, while guiding the team to ensure project success.
Key Responsibilities:
- Lead, mentor, and manage a team of Data Engineers to deliver high-quality data solutions.
- Design, develop, and implement cloud-based data pipelines, ETL processes, and data integration solutions using platforms AWS.
- Collaborate with cross-functional teams to understand data requirements and design scalable data models and architectures that align with BFSI industry needs.
- Optimize data storage, processing, and retrieval for maximum performance and cost-efficiency in cloud environments.
- Implement data security and compliance measures to ensure the protection and integrity of sensitive BFSI data.
- Work closely with data scientists and analysts to enable seamless access to high-quality data for analytical purposes.
- Troubleshoot and resolve data-related issues, ensuring data availability and reliability for BFSI customers.
- Stay updated on industry best practices, emerging cloud technologies, and trends in the BFSI sector to drive continuous improvement and innovation.
- Manage project timelines, delegate tasks, and ensure the successful delivery of data engineering projects.
Qualifications & Experience:
- Minimum 6 years experience with good Hands on Pyspark, Airflow, AWS.
- Advanced SQL and PySpark expertise; strong understanding of Data Engineering concepts (DW, Data Lake, OLTP vs OLAP, etc.)
- API development experience (preferably in Python)
- Familiarity with Docker and Kubernetes
- Experience with Airflow
- Exposure to Hudi, Iceberg, or Delta Lake
- Strong AWS project experience (should also be adaptable to other cloud platforms if required)
- Proven experience in leading and managing a team of Data Engineers in previous roles.
- Bachelor's degree in Engineering, or a related field
- Proficiency in programming languages like Python, Java, or spark for data processing and ETL operations.
- Experience with big data technologies (e.g., Hadoop, Spark) and data processing frameworks.
- Familiarity with BFSI industry data standards, compliance requirements, and security protocols.
- Excellent problem-solving skills and the ability to work in a collaborative, cross-functional team environment
