Data Engineer

Job Description – Data Engineer

Job Title: Data Engineer

Experience: 4–10 Years

Location: Chennai

Notice Period: Immediate to 15 Days

Work Mode: Hybrid

Mandatory Skills: Azure, Databricks, Python, PySpark, SQL

Additional Requirement: Minimum 3 months of GenAI experience

Role Overview

Altimetrik is looking for an experienced Data Engineer with strong expertise in Azure, Databricks, and big data processing frameworks. The ideal candidate will be proficient in building scalable data pipelines, transforming large datasets, and enabling analytics. Hands-on experience with Generative AI (GenAI) is required to support emerging AI-driven initiatives.

Key Responsibilities

• Data Pipeline Development: Design, build, and optimize scalable data pipelines using Azure Data Engineering services and Databricks.
• Big Data Processing: Develop and maintain large-scale data transformations using PySpark, ensuring performance, reliability, and efficiency.
• Data Integration: Integrate structured and unstructured data from various sources using SQL, Python, and Azure data services.
• Azure Platform Expertise: Work extensively with Azure (ADF, ADLS, Azure Databricks, Azure Functions, etc.) to enable seamless data workflows.
• GenAI Integration: Apply Generative AI techniques (minimum 3 months) for tasks such as data enrichment, automation, code generation, and advanced analytics.
• Data Modeling: Design and implement robust data models supporting both operational and analytical requirements.
• Performance Optimization: Improve processing time, reduce data pipeline latency, and ensure the scalability of data workloads.
• Collaboration: Work closely with data scientists, analysts, and business teams to deliver high-quality data solutions.
• Quality & Governance: Implement data quality checks, best coding practices, and governance standards across all pipelines.
• Documentation: Maintain clear and comprehensive technical documentation for pipelines, workflows, and engineering processes.

Required Skills & Qualifications

• 4–10 years of experience as a Data Engineer or in a similar role.
• Strong hands-on experience with Azure and Azure Databricks.
• Advanced proficiency in Python, PySpark, and SQL.
• Experience designing efficient ETL/ELT pipelines using cloud-native tools.
• Good understanding of data warehousing concepts and distributed data systems.
• Minimum 3 months of hands-on GenAI exposure (e.g., LLMs, embeddings, RAG, AI-assisted data transformations).
• Strong analytical and problem-solving skills.
• Excellent communication skills and ability to collaborate with cross-functional teams.
• Ability to work in a fast-paced environment with tight timelines.