Data Engineer

5-14 Years

This job is no longer accepting applications

Job Description

Greetings from TCS!

Job Title: Azure Data Engineer

Required Skillset: AWS(preferable)/AZURE Databricks

Location: PAN INDIA

Experience Range: 5-14 years

Job Description:

Must Have:

Experience: More than 4 years in data integration, pipeline development, and data warehousing, with a strong focus on AWS/AZURE Databricks.
Proficiency in Databricks platform, management, and optimization.
Strong experience in AWS/AZURE Cloud, particularly in data engineering and administration, with expertise in Apache Spark/Azure Synapse Analytics Spark pool, S3, Athena, Glue/AZURE Data factory, Kafka, Lambda/Azure functions, Redshift, and RDS.
Proven experience in data engineering performance tuning and analytical understanding in business and program contexts.
Solid experience in Python development, specifically in pySpark within the AWS/AZURE Cloud environment, including experience with Terraform.
Knowledge of databases (Oracle, SQL Server, PostgreSQL, Redshift, MySQL, or similar) and advanced database querying.
Experience with source control systems (Git, Bitbucket) and Jenkins for build and continuous integration.
Understanding of continuous deployment (CI/CD) processes.
Experience with Airflow and additional Apache Spark knowledge is advantageous.
Exposure to ETL tools, including Informatica.

Good To Have:

Key Responsibilities:

Administer, manage, and optimize the Databricks environment to ensure efficient data processing and pipeline development.
Perform advanced troubleshooting, query optimization, and performance tuning in a Databricks environment.
Collaborate with development teams to guide, optimize, and refine data solutions within the Databricks ecosystem.
Ensure high performance in data handling and processing, including the optimization of Databricks jobs and clusters.
Engage with and support business teams to deliver data and analytics projects effectively.

Thanks & Regards,

Ria Aarthi A.