Databricks Engineer

Job Description:

• We are looking for an experienced Databricks Engineer with 5 8 years of experience in designing developing and optimizing large scale data solutions
• The ideal candidate should have strong expertise in Databricks PySpark Spark SQL cloud platforms AWS Azure GCP and modern data engineering practices to build scalable and high performance data pipelines

Key Responsibilities:

• Roles Responsibilities
• Design develop and maintain data pipelines using Databricks
• Build ETL ELT workflows for large scale data processing
• Develop and optimize data transformation logic using PySpark and Spark SQL
• Implement and manage Delta Lake for reliable and efficient data storage
• Work with batch and real time data ingestion frameworks
• Monitor and optimize data pipelines for performance and scalability
• Collaborate with data analysts data scientists and business stakeholders
• Ensure data quality security and governance standards are followed
• Troubleshoot production issues and provide ongoing support

Technical Requirements:

• 5 8 years of experience in Data Engineering and Big Data technologies
• Strong hands on experience with Databricks PySpark and Spark SQL
• Experience with Delta Lake Databricks Workflows and Unity Catalog
• Proficiency in Python and SQL
• Experience with Azure AWS GCP cloud platforms
• Knowledge of Azure Data Factory ADF Apache Airflow or similar orchestration tools
• Understanding of data warehousing concepts and ETL ELT methodologies
• Experience with Kafka Event Hub for streaming data processing is preferred
• Familiarity with Git CI CD pipelines and DevOps practices
• Strong problem solving analytical and communication skills

Preferred Skills: