Sr Data Engineer

Key Responsibilities

Design, develop, and maintain scalable and efficient data pipelines using Python and PySpark.

Build and optimize ETL/ELT workflows on Databricks.

Write complex, high-performance queries using SQL and Spark SQL.

Process and transform large-scale structured and semi-structured datasets.

Ensure data quality, reliability, and performance across all data pipelines.

Collaborate with data analysts, data scientists, and business stakeholders to understand and deliver on data requirements.

Optimize data processing jobs for performance, cost efficiency, and scalability.

Implement best practices for data engineering, including version control, testing, and documentation.

Troubleshoot and resolve data-related issues in production environments.

Participate in code reviews and mentor junior data engineers.