Senior Data Engineer

We are looking for a Senior Data Engineer to design and build scalable, reliable batch and streaming data platforms. The role involves developing production-grade ETL/ELT pipelines, data ingestion, transformation, validation, reconciliation, and publishing workflows. The candidate will work extensively with Python, SQL, Apache Spark/PySpark, and AWS to process large-scale data.

Key responsibilities include designing cloud data lakes, lakehouses and data warehouse components; implementing incremental loading, CDC, schema evolution and idempotent processing; and optimizing Spark jobs, SQL queries, partitioning and cloud resources. The role also involves implementing data-quality checks, monitoring, alerting, orchestration, recovery mechanisms and operational processes.

Requirements: 5+ years of relevant Data Engineering experience, strong Python and advanced SQL skills, and mandatory hands-on Apache Spark experience including performance tuning. Strong knowledge of data modelling, data quality, orchestration and production data pipelines is required. Experience with AWS cloud platforms is essential.

Skills: apache spark,python,aws