k, with experience building high-performance ETL/ELT pipelines and working with large-scale structured and unstructured datasets.

This role involves close collaboration with analytics, reporting, and business teams to enable data-driven decision-making.

Key Responsibilities

Design, develop, and implement Databricks-based data engineering solutions
Build scalable ETL/ELT pipelines using Apache Spark
Ingest, transform, and validate large volumes of structured and unstructured data
Optimize Spark jobs for performance, scalability, and cost efficiency
Work with cloud-based data lakes and modern data platforms
Ensure adherence to data quality, governance, and security standards
Collaborate with analytics and reporting teams to support downstream use cases
Support implementation, migration, and modernization of data platforms

Technical Requirements (Must-Have)

Strong hands-on experience with Databricks
Proficiency in : Apache Spark SQL Python
Experience working with cloud-based data platforms (AWS / Azure / GCP)
Experience with data lakes and large-scale data processing
Strong understanding of data modeling and analytics workflows

Good-to-Have Skills

Experience with Delta Lake
Exposure to streaming frameworks (Kafka / Structured Streaming)
Knowledge of CI/CD and DevOps practices in data engineering
Experience with BI/reporting tools (Power BI, Tableau, etc)

(ref:hirist.tech)

GyanSys - Databricks Consultant/Senior Consultant/Lead - Data Engineering Solutions