Data Engineer

Hiring: Data Engineer (Python + PySpark + Databricks/Snowflake)

Experience: 8 Years

Location: India

Notice Period: Immediate to 30 Days Preferred

Employment Type: Full-Time

Job Description

We are looking for a skilled Data Engineer with strong expertise in Python, PySpark, and modern cloud-based data platforms. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines and data processing solutions to support business analytics and data-driven decision-making.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Python and PySpark.
• Build and optimize batch and streaming data processing workflows.
• Work with cloud data platforms such as Databricks and Snowflake.
• Implement Medallion Architecture (Bronze, Silver, Gold layers) and data transformation best practices.
• Monitor, troubleshoot, and optimize Spark jobs for performance and cost efficiency.
• Configure autoscaling and cluster optimization strategies.
• Collaborate with business, analytics, and engineering teams to deliver reliable data solutions.
• Implement CI/CD best practices and version control processes.
• Participate in data platform modernization and cloud migration initiatives.

Required Skills

• Strong hands-on experience with Python and PySpark.
• Solid understanding of Apache Spark architecture and performance tuning.
• Experience with Databricks, Snowflake, or similar cloud data platforms.
• Strong knowledge of ETL/ELT and data pipeline development.
• Experience with batch and streaming data processing.
• Understanding of Medallion Architecture.
• Knowledge of cluster optimization and autoscaling.
• Experience with Git, CI/CD, and DevOps practices.

Preferred Skills

• Terraform or Infrastructure as Code (IaC).
• Experience with AWS, Azure, or GCP.
• Exposure to workflow orchestration tools such as Airflow.