We are looking for a skilled Databricks Engineer / Data Integration Developer with strong experience in Python, PySpark, SQL, and Snowflake to design, develop, and optimize scalable data pipelines and ETL/ELT solutions. The ideal candidate should have hands-on experience with Databricks and cloud-based data platforms.
Key Responsibilities
- Design, develop, and maintain data pipelines using Databricks, PySpark, and Python.
- Build and optimize ETL/ELT workflows for large-scale data processing.
- Develop and manage data integration solutions between multiple source systems and Snowflake.
- Write complex SQL queries, stored procedures, and performance tuning activities.
- Implement data quality checks, monitoring, and troubleshooting processes.
- Collaborate with business stakeholders, data analysts, and engineering teams to deliver data solutions.
- Optimize data models and ensure high-performance data processing.
- Support production deployments and resolve data-related issues.
Required Skills
- Strong experience in Python, PySpark, and SQL.
- Hands-on experience with Databricks development and administration.
- Experience working with Snowflake Data Warehouse.
- Good understanding of ETL/ELT concepts and Data Integration.
- Experience with data modeling, performance tuning, and optimization.
- Knowledge of cloud platforms such as AWS, Azure, or GCP.
- Familiarity with Git, CI/CD, and Agile methodologies.
Preferred Skills
- Experience with workflow orchestration tools such as Airflow or Databricks Workflows.
- Knowledge of Delta Lake, Data Lakehouse architecture.
- Exposure to real-time data processing and streaming technologies.
