Senior Data Engineer

We are seeking a Senior Data Engineer with 6+ years of hands-on experience in Spark and Spark SQL, strong SQL expertise, and solid batch processing experience.

The role involves building and optimizing data pipelines to support analytics and business needs, ensuring data reliability and performance, and collaborating effectively with cross-functional teams.

Must Have skills:

Spark, Spark SQL, at least 6+ years working experience with spark.

Very Strong SQL, Batch Exp is a must have. Good Project Experience.

Python is a plus not a must have.

Participate in code reviews, design discussions, and agile delivery processes.

Required Skills & Experience:

• 6 years of hands-on experience with Apache Spark (especially Spark SQL and batch processing).
• Strong SQL expertise — ability to write efficient, optimized, and complex SQL queries for large datasets.
• Solid understanding of data pipeline fundamentals, including ingestion, transformation, and loading (ETL/ELT) processes.
• Must have experience with batch data processing (scheduling, dependencies, failure recovery, performance tuning).
• Knowledge of data warehousing concepts and familiarity with relational databases.
• Experience working with structured and semi-structured data (CSV, Parquet, JSON, etc.).
• Strong problem-solving and debugging skills, particularly around pipeline performance and data quality.