About Birlasoft:

Birlasoft, a global leader at the forefront of Cloud, AI, and Digital technologies, seamlessly blends domain expertise with enterprise solutions. The company's consultative and design-thinking approach empowers societies worldwide, enhancing the efficiency and productivity of businesses. As part of the multibillion-dollar diversified CKA Birla Group, Birlasoft with its 12,000+ professionals, is committed to continuing the Group's 170-year heritage of building sustainable communities.

Job Title – Lead Data Engineer / ETL Lead

Experience – 6-10 Years

Location – Any BSL

Education – B.E/B.Tech

Job Summary

We are seeking a highly experienced Lead Data Engineer to design, build, and lead large-scale data integration, migration, and analytics platforms. This role requires deep expertise in enterprise ETL tools, modern cloud-native data architectures, and real-time data processing, along with strong leadership and delivery management skills.

Key Responsibilities

Lead the design, development, and optimization of end-to-end data pipelines across batch and real-time processing
Architect and implement enterprise-grade ETL solutions using Informatica, Ab Initio, and cloud-native services
Drive large-scale data migration and conversion initiatives, including mock runs, reconciliation, validation, and production cutovers
Design and manage cloud-based data platforms, leveraging Snowflake and AWS analytics services
Build and optimize PySpark-based data processing frameworks for high-volume datasets
Implement real-time data ingestion and transformation pipelines using Kafka and streaming technologies
Own performance tuning, scalability, cost optimization, and SLA adherence for data workloads
Collaborate closely with business, functional, and architecture teams to translate requirements into robust technical solutions
Lead and mentor development teams, conducting code reviews and enforcing engineering best practices
Oversee CI/CD, scheduling, monitoring, and operational stability of data pipelines
Support Agile delivery by participating in planning, backlog grooming, execution, and retrospectives

Required Skills & Qualifications

Core Data Engineering

Strong hands-on experience with Ab Initio and Informatica PowerCenter / IICS
Advanced expertise in data warehousing concepts, dimensional modeling, and ETL design patterns
Proven experience with Snowflake data platform capabilities

Cloud & Big Data

Extensive hands-on experience with AWS data services (Glue, Redshift, Athena, EMR, S3, CI/CD pipelines)
Proficiency in PySpark, Spark SQL, Hive, and Hadoop ecosystems
Experience with Kafka-based streaming architectures and KSQL

Programming & Databases

Advanced SQL and PL/SQL
Strong Python and UNIX shell scripting skills for automation
Experience optimizing large-scale database and ETL workloads

Delivery & Leadership

Experience leading multi‑team, offshore‑onsite delivery models
Strong Agile/Scrum execution experience
Excellent communication, stakeholder management, and problem-solving skills

Preferred Certifications

Cloud and database-related professional certifications
ETL or data platform certifications