We are looking for a skilled Data Engineer with strong hands-on experience in Databricks and modern data engineering technologies. The ideal candidate should be proficient in building scalable ETL pipelines, working with large datasets, and ensuring data quality, reliability, and performance across enterprise data platforms.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines using Databricks and Apache Spark.
- Work extensively with SQL and Python for data transformation, analysis, and automation.
- Build and optimize data processing workflows on cloud-based analytic data stores such as Databricks.
- Develop and manage data ingestion from multiple structured and unstructured data sources.
- Ensure data quality, reliability, integrity, and performance across data platforms.
- Work with RDBMS systems and optimize complex SQL queries.
- Collaborate with cross-functional teams including analytics, business, and application teams.
- Monitor and troubleshoot production data pipelines and workflows.
- Implement best practices for data governance, security, and scalability.
Required Skills
- Strong hands-on experience with Databricks
- Expertise in SQL and Python
- Strong experience in ETL/ELT development
- Hands-on experience with Apache Spark
- Good understanding of RDBMS concepts
- Experience working with Analytical Data Stores/Data Warehousing
- Strong understanding of data quality, validation, and reliability frameworks
- Experience in performance tuning and optimization of data pipelines
Preferred Skills
- Experience with cloud platforms like AWS, Azure, or GCP
- Knowledge of Delta Lake, Data Lake architecture
- Exposure to CI/CD and DevOps practices in data engineering
- Experience with orchestration tools like Airflow or ADF
Qualifications
- Bachelor's degree in Computer Science, Information Technology, or related field
- Strong analytical and problem-solving skills
- Good communication and collaboration abilities
Skills: rdbms,apache spark,databricks
