Azure Data Engineer

Greetings from TCS!!

TATA Consultancy Services is hiring for Azure Databricks Data Engineer

Desired experience range: 6+ years

Job location: Chennai / Kolkata / Pune / Hyderabad / Bangalore

Job description:

Desired Competencies (Technical/Behavioral Competency)

Must-Have

• 6+ years of experience in data engineering with strong expertise in Azure Data Lake, Databricks, and ADF.
• Hands-on expertise in SQL and PySpark or Scala for building complex ETL/ELT solutions.
• Proven track record with batch & real-time ingestion, Kafka, and large-scale big data processing.
• Experience with relational and NoSQL engines: SQL Server, Oracle, Azure SQL, and Cosmos DB.
• Experience with Informatica PowerCenter, Informatica IDMC, or equivalent cloud data integration tools.

Good-to-Have

• Insurance domain knowledge (Policy, Claims, Billing, Underwriting, Actuarial, Regulatory data).
• Experience with Azure DevOps (AzDO) and GitHub-based CI/CD pipelines.
• Practical usage of Copilot for data engineering and development productivity.
• Exposure to Generative AI / GenAI use cases in data platforms and analytics.
• Familiarity with Maestro or similar enterprise schedulers, and data tools like Precisely or equivalent.

Responsibilities / Expectations from the Role

1. Design and build batch and streaming data pipelines using Azure, Databricks, and Kafka.
2. Develop highly scalable ETL/ELT solutions using SQL, PySpark, or Scala.
3. Implement and support robust Data Lake / Lakehouse architectures.
4. Integrate and transform structured and unstructured data from legacy and modern platforms.
5. Apply rigorous data quality checks and support data governance basics including metadata, data lineage, and access controls.
6. Optimize Spark jobs systematically for performance tuning, execution reliability, and cost efficiency.
7. Collaborate dynamically with cross-functional business teams and mentor junior engineers within the ecosystem.