Data Engineer
Job Description
Role: Data Engineer
Location: Mumbai/ Pune/ Noida/ Bangalore/ Kochi
Key Responsibilities & Requirements:
Core Engineering: Hands-on experience developing large-scale Apache Spark pipelines using Python.
AWS Ecosystem: Deep fluency across S3, Glue Data Catalog, Athena, EMR, and Redshift.
Data Quality & Reliability: Strong focus on building production-grade ingestion, transformation, and automated validation frameworks at scale.
Highly Preferred:
- AWS Lake Formation
- Architecture & Governance: End-to-end Data Lake design and permission-model test validation.
More Info
Key Skills
AWS Lake Formation
Data Lake design
Athena
About Company
Skills:
Apache Airflow, Hadoop, Spark, Apache Beam, Python, Google DataProc, dbt, Google DataFlow, Google Cloud Ecosystem, Google Cloud Storage
Skills:
snowflake , Apache Airflow, Pyspark, Amazon Kinesis, Apache Kafka, Azure, Python, Delta Lake on Databricks, Databricks Workflows, AWS CloudShell
Skills:
Gold layers), dbt (Data Build Tool), snowflake , workflow management , Medallion Architecture (Bronze, Performance Tuning, Pl Sql, Data Quality, Data Modeling, Data Governance, Data Warehousing, Sql, Dimensional Modeling, Data Transformation, Query Optimization, ETL/ELT pipelines, silver, Troubleshooting, Cloud-based data platforms, Warehouse optimization
Skills:
Azure Data Lake Storage (ADLS), Pyspark, Scala, Azure Databricks, Sql, Azure Data Factory, Azure Synapse Analytics, Spark, Shell scripting, Cosmos DB, Python, Azure DevOps, Spark Structured Streaming, Azure SQL Data Warehouse, Delta Lake
Skills:
retrieval-augmented generation (RAG) pipelines, Data Transformation, Pyspark, Sql, Git, Pandas, Docker, FastAPI, Python, agentic AI systems, LangChain, vector databases, Azure AI Agent Service, embedding-based retrieval, LangGraph, multi-agent workflows, LLM-powered applications, CI/CD