Technical Lead - Data Engineering

- - - - - - - - - - - -

Job Title: Technical Lead - Data & AI Delivery

Experience: 10-15 Years

Role Type:Technical Leadership & Delivery Management

Role Objective

• We are seeking a high-caliber Technical Lead to spearhead the delivery of complex Data and AI projects.
• You will convert strategic AI roadmaps into scalable production realities by blending expert data engineering with the precision and discipline of modern software engineering.

Key Responsibilities

• Strategic Delivery Management: Lead the end-to-end lifecycle of Data/AI projects, translating business goals into actionable technical roadmaps and AI strategies.
• Architectural Excellence: Design and implement Medallion Data Lake architectures on Azure, ensuring seamless integration of structured and unstructured data.
• Hands-on Optimization: Act as the subject matter expert in PySpark and Databricks, directly overseeing the optimization of high-performance data pipelines and ETL/ELT patterns.
• AI Integration: Collaborate with Data Scientists to integrate models into production environments, ensuring smooth model operations and deployment.
• Governance & Security: Enforce strict data security and compliance standards.
• Technical Leadership: Guide a cross-functional squad of engineers and scientists through complex technical roadblocks, fostering a culture of excellence and continuous learning.

Technical Skills & Qualifications:

Core Tech Stack (Must-Have)

• Languages: Expert-levelPySpark, Databricks, Python, SQL.
• Expertise in Azure (Azure Data Factory, Databricks, Azure Storage).
• Data Architecture: Mastery of Data Modeling, Data Warehousing, and Medallion (Bronze/Silver/Gold) patterns.
•  Good understanding onData Ingestion Frameworks and practices.
•   Databases: Snowflake, Azure Datalake , Lakehouse delta lake
•   Understanding of LLM framework will be a plus.

AI & Machine Learning Big Plus:

• Proficiency in ML/DL frameworks and the development-to-deployment lifecycle.
• Strong experience integrating AI models with Spark-based data processing.
•   Experience on integration with LLM models and storage will be plus.

Integration & Infrastructure:

• Strong understanding of Cloud Computing and Azur services.

Soft Skills & Leadership:

• Communication: Ability to articulate complex technical concepts to both engineering teams and non-technical stakeholders.
• Problem-Solving: A strategic thinker with a proven track record of resolving architectural bottlenecks.
• Collaborative Leadership: Ability to motivate teams and drive initiatives forward within a fast-paced corporate environment.
• Industry Awareness: Proactive engagement with emerging trends in Data Engineering, Generative AI, and Cloud Architecture.
• ContinuousLearning.

Preferred Experience:

• Prior experience in a lead or co-lead role for Data or AI product development .
•   Proven expertise in handling large-scale migrations to Azure-native data stacks.