Data Engineer

We are seeking a highly skilled Data Engineer with 7+ years of hands-on experience in designing, building, and maintaining scalable data pipelines and data platforms. The ideal candidate should have strong expertise in Python, PySpark, SQL, and Spark, along with excellent analytical and communication skills. The role requires the ability to work independently while collaborating with cross-functional teams to deliver reliable and high-quality data solutions.

Key Responsibilities

Design, develop, and maintain scalable data pipelines and ETL/ELT processes

Build and optimize large-scale data processing systems using Spark/PySpark

Develop efficient and complex SQL queries for data extraction and transformation

Ensure data quality, integrity, and consistency across systems

Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements

Monitor, troubleshoot, and optimize data workflows and performance issues

Implement data validation, error handling, and logging mechanisms

Work with structured and unstructured data from multiple sources

Maintain documentation for data architecture, pipelines, and processes

Participate in code reviews and follow best coding practices

Required Skills & Qualifications

Minimum 7+ years of experience in Data Engineering or related roles

Experience in Client Handling with good communication

Strong proficiency in Python / PySpark

Advanced knowledge of SQL and database concepts

Hands-on experience with Apache Spark

Solid data analysis and problem-solving skills

Excellent communication skills (written and verbal)

Ability to work independently and manage tasks effectively

Understanding of data warehousing concepts and ETL methodologies

Experience with version control systems such as Git

Good to Have (Preferred Skills)

Experience with cloud platforms (AWS-recommended)

Knowledge of big data technologies such as Hadoop, Hive, or Kafka

Exposure to orchestration tools like Airflow or similar

Experience with data modeling and schema design

Familiarity with CI/CD pipelines and DevOps practices

Understanding of data security and governance principles

Soft Skills

Strong analytical and critical-thinking abilities

Effective collaboration and teamwork skills

Self-motivated with strong ownership mindset

Ability to manage multiple priorities in a fast-paced environment