Data Engineering Lead

Join Verdantas – A Top #ENR 81 Firm,

We at Verdantas, seeking a Data Engineer Lead in Pune to architect and implement scalable, secure AI/ML data solutions aligned with business objectives. The role involves designing robust batch and real-time data architectures, leading ETL/ELT pipeline development, managing data lakes and warehouses, ensuring data quality and governance, mentoring data engineers, collaborating with stakeholders, and deploying cloud-based data infrastructure with CI/CD and cost optimization. Candidates should have 8+ years in data engineering with leadership experience, strong skills in SQL, Python, distributed processing frameworks, and cloud-native data services, with preferred certifications including Google Professional Data Engineer, Azure Data Engineer Associate, and AWS Certified Data Analytics Specialty. The position requires in-office work in Pune, Maharashtra, India.

Key Responsibilities

Architecture & Strategy

• Design and implement robust data architectures (batch and real-time) using modern data platforms.
• Define data engineering standards, best practices, and governance policies.
• Evaluate and recommend tools and technologies for data ingestion, storage, processing, and orchestration.

Development & Operations

• Lead the development of ETL/ELT pipelines using tools like Apache Spark, Airflow, dbt, or Azure Data Factory.
• Build and manage data lakes, data warehouses, and data marts.
• Ensure data quality, lineage, and observability across pipelines.

Team Leadership

• Mentor and guide data engineers, fostering a culture of technical excellence and continuous improvement.
• Collaborate with data scientists, analysts, and business stakeholders to understand data needs and deliver solutions.
• Manage project timelines, resource allocation, and delivery milestones.

Cloud & DevOps Integration

• Deploy and manage data infrastructure on cloud platforms (AWS, Azure, GCP).
• Implement CI/CD pipelines for data workflows and infrastructure as code (IaC).
• Optimize performance and cost of data systems.

Skills & Qualifications

• Bachelor's or master's in computer science, Data Engineering, or related field.
• 8+ years of experience in data engineering, with 2+ years in a leadership role.
• Strong expertise in SQL, Python, and distributed data processing frameworks (Spark, Flink).
• Experience with cloud-native data services (e.g., AWS Glue, Azure Synapse, BigQuery).
• Familiarity with data modeling, data governance, and metadata management.
• Excellent problem-solving, communication, and stakeholder management skills.

Preferred Certifications:

• Google Professional Data Engineer
• Microsoft Certified: Azure Data Engineer Associate
• AWS Certified Data Analytics – Specialty

Location and Work Set-up

· Pune, Maharashtra, India

· Work Mode: In Office