Data Engineer
- Posted 5 hours ago
- Be among the first 10 applicants
Job Description
Data Engineer (4–5 Years Experience)
Location: Chennai
Experience: 4–5 Years
Employment Type: Full-time
Job Summary
We are seeking a Data Engineer with 4–5 years of experience to build and maintain scalable data pipelines and data integration solutions. The ideal candidate should have strong expertise in PySpark, Databricks, Azure Data Factory (ADF), Python, and SQL, with hands-on experience in cloud-based data platforms.
Key Responsibilities
* Design and develop scalable ETL/ELT pipelines using PySpark and Databricks
* Build and manage data workflows using Azure Data Factory (ADF)
* Integrate data from multiple sources such as APIs, databases, flat files, and cloud storage
* Optimize and maintain large-scale data processing systems
* Ensure data quality, validation, and governance across pipelines
* Collaborate with cross-functional teams including data scientists and analysts
* Monitor, troubleshoot, and improve pipeline performance
* Implement reusable frameworks and best practices for data engineering
* Support deployment using CI/CD pipelines
Required Skills
* Strong experience with Azure Data Factory (ADF) for pipeline orchestration
* Hands-on experience with Databricks and PySpark
* Proficiency in Python
* Advanced SQL skills including complex queries and performance tuning
* Experience working on Azure Cloud (ADLS, Azure SQL, etc.)
* Good understanding of data warehousing and data modeling concepts
Preferred Qualifications
* Experience with Azure ecosystem including ADF, ADLS, Synapse, and Key Vault
* Knowledge of workflow orchestration and scheduling tools
* Familiarity with Delta Lake architecture
* Experience handling large datasets and performance tuning
Nice to Have
* Experience with streaming tools such as Kafka or Event Hubs
* Knowledge of DevOps practices and CI/CD using Azure DevOps
* Exposure to data governance and security practices
Soft Skills
* Strong analytical and problem-solving skills
* Good communication and teamwork abilities
* Ability to work in a fast-paced environment
Interested candidates can also apply at [Confidential Information]
Location: Chennai
Experience: 4–5 Years
Employment Type: Full-time
Job Summary
We are seeking a Data Engineer with 4–5 years of experience to build and maintain scalable data pipelines and data integration solutions. The ideal candidate should have strong expertise in PySpark, Databricks, Azure Data Factory (ADF), Python, and SQL, with hands-on experience in cloud-based data platforms.
Key Responsibilities
* Design and develop scalable ETL/ELT pipelines using PySpark and Databricks
* Build and manage data workflows using Azure Data Factory (ADF)
* Integrate data from multiple sources such as APIs, databases, flat files, and cloud storage
* Optimize and maintain large-scale data processing systems
* Ensure data quality, validation, and governance across pipelines
* Collaborate with cross-functional teams including data scientists and analysts
* Monitor, troubleshoot, and improve pipeline performance
* Implement reusable frameworks and best practices for data engineering
* Support deployment using CI/CD pipelines
Required Skills
* Strong experience with Azure Data Factory (ADF) for pipeline orchestration
* Hands-on experience with Databricks and PySpark
* Proficiency in Python
* Advanced SQL skills including complex queries and performance tuning
* Experience working on Azure Cloud (ADLS, Azure SQL, etc.)
* Good understanding of data warehousing and data modeling concepts
Preferred Qualifications
* Experience with Azure ecosystem including ADF, ADLS, Synapse, and Key Vault
* Knowledge of workflow orchestration and scheduling tools
* Familiarity with Delta Lake architecture
* Experience handling large datasets and performance tuning
Nice to Have
* Experience with streaming tools such as Kafka or Event Hubs
* Knowledge of DevOps practices and CI/CD using Azure DevOps
* Exposure to data governance and security practices
Soft Skills
* Strong analytical and problem-solving skills
* Good communication and teamwork abilities
* Ability to work in a fast-paced environment
Interested candidates can also apply at [Confidential Information]
More Info
Key Skills
About Company
Skills:
Aws Lambda, Data Quality, Metadata Management, Amazon Kinesis, AWS Glue, AWS IAM, Databricks, AWS, Apache Iceberg, AI Tools, Delta Lake
Skills:
Pyspark, Scala, Azure Databricks, Informatica, Sql, ELT, Apache Airflow, Azure Data Factory, Azure Data Lake, Python, Etl, Oracle PLSQL, Unity Catalog, CI CD pipelines, Delta Lake
Data Engineer (PySpark, Informatica BDM, ETL, Banking)Insights Techhub International Private Limited
Skills:
Informatica BDM (Big Data Management), Pyspark, Etl, Banking or Financial Services domain
Skills:
Pyspark, Apache Spark, Data Warehousing, Dimensional Modeling, Sql, ELT, Git, Gcp, Azure, Python, AWS, Etl, Generative AI, CI CD, Large Language Models
Skills:
Py Spark, Spark, Python, Sql, Databricks