Job Description:

We are seeking a highly motivated Databricks Data Engineer to design develop and optimize modern data platforms leveraging Databricks and cloud based technologies
The ideal candidate will have hands on experience in building large scale data pipelines implementing Lakehouse architectures and enabling advanced analytics solutions using Databricks Spark and cloud ecosystems
In this role you will work closely with data architects analysts business stakeholders and engineering teams to deliver scalable high performance data solutions that support enterprise wide digital transformation initiatives

Key Responsibilities:

Design develop and maintain scalable data pipelines using Databricks and Apache Spark
Build and optimize ETL ELT workflows for processing large scale structured and unstructured datasets
Develop data ingestion frameworks to integrate data from multiple sources including databases APIs applications and cloud storage
Implement and manage Lakehouse architecture using Databricks and Delta Lake
Optimize Spark jobs and Databricks workloads for performance scalability and cost efficiency
Collaborate with analytics BI and business teams to understand data requirements and deliver reusable data assets
Implement data quality frameworks monitoring solutions and governance controls across data pipelines
Troubleshoot production issues and provide timely resolution to ensure business continuity
Participate in architecture reviews code reviews and technical discussions to promote best practices
Create and maintain technical documentation reusable components and deployment standards

11 years of Strong hands on experience with Databricks
Proficiency in PySpark or Spark SQL for distributed data processing
Strong SQL skills with experience in query optimization and performance tuning
Experience in designing and developing ETL ELT pipelines
Hands on experience with Delta Lake DataFrames and Spark architecture
Knowledge of data warehousing concepts dimensional modeling and data lake architectures
Experience working with cloud platforms such as Azure AWS or GCP
Familiarity with Git based source code management and DevOps practices
Strong analytical debugging and problem solving skills

Infosys Data Analytics practice helps organizations modernize their data landscape through cloud native architectures advanced analytics AI and intelligent data platforms
As a Databricks Data Engineer you will work on enterprise scale data modernization programs involving Lakehouse implementations cloud migrations real time analytics and AI ready data ecosystems

Technology->Data Engineering->Databricks