Databricks, PySpark, Python - Data Engineer

Job Description:

• We are seeking a highly skilled Team Lead Data Engineering with 7 9 years of experience in designing and developing scalable data solutions using Databricks PySpark and Python
• The candidate will be responsible for leading a small team driving technical delivery and building high performance data pipelines for enterprise analytics platforms

Key Responsibilities:

• Lead the design development and deployment of data engineering solutions using Databricks and PySpark
• Develop scalable ETL ELT pipelines and data transformation frameworks
• Write efficient Python and SQL code for data processing and analytics
• Work closely with stakeholders to understand business requirements and translate them into technical solutions
• Optimize Spark jobs for performance scalability and cost efficiency
• Ensure data quality governance and security standards are followed
• Review code mentor team members and provide technical guidance
• Support production deployments incident troubleshooting and root cause analysis
• Participate in Agile ceremonies and drive sprint deliverables

Technical Requirements:

• 7 9 years of overall IT experience
• Strong hands on expertise in Databricks and PySpark
• Proficiency in Python programming and SQL
• Experience building enterprise scale data pipelines and data lake solutions
• Strong understanding of distributed computing concepts and Spark architecture
• Experience in performance tuning and optimization
• Excellent stakeholder management and team coordination skills

Preferred Skills:

Technology->Data Engineering->Databricks,Technology->AI-Data science->PYTHON,Technology->Big Data - Data Processing->PySpark