Databricks Data Engineer

Job description:

We are seeking a highly experienced Senior Databricks Data Engineer to design, build, and optimize our data lake house environment. You will leverage your 10+ years of expertise to develop complex data pipelines, ensure data quality, and drive innovation using Databricks and Apache Spark.

Responsibilities:

• Design and implement scalable and robust data pipelines using Databricks and Apache Spark.

• Architect and optimize data lake house solutions for performance and reliability.

• Develop and maintain complex ETL/ELT processes for large-scale data ingestion and transformation.

• Write and optimize high-performance SQL and Spark code for data processing and analysis.

• Implement data quality checks, data validation, and data governance practices.

• Troubleshoot and resolve complex data processing issues and performance bottlenecks.

• Collaborate with data scientists, analysts, and other engineering teams to deliver data-driven solutions.

• Provide technical leadership and mentorship to junior engineers.