Databricks Data Engineer (Spark/SAP)
Type: Full-time
Location- Remote
Experience- 4+ years
About the role
We are hiring a Databricks Data Engineer to build the pipelines at the heart of a large-scale predictive analytics programme. Working under a Lead Data Engineer, you will develop, test, and operate automated Databricks pipelines that extract data from SAP and other enterprise systems, apply quality checks, and serve curated datasets to ML and BI workloads.
What you'll do
- Build automated ingestion and transformation pipelines in Spark/PySpark on Databricks
- Develop SAP data extraction jobs and integrate additional enterprise sources (planning, scheduling, operational systems)
- Implement data quality rules, anomaly detection checks, and lineage metadata in every pipeline
- Orchestrate and schedule workloads; monitor, troubleshoot, and tune pipeline runs to meet contractual reliability targets
- Support SIT/UAT with test data preparation and defect resolution
- Produce clear technical documentation for handover
What you'll need
- 4+ years in data engineering with strong Spark/PySpark and SQL
- Hands-on experience extracting data from SAP (any of ECC, S/4HANA, MM, PM) or comparable ERP systems
- Experience with pipeline orchestration tools and production operations
- Working knowledge of Delta Lake or comparable lakehouse formats
- A quality-first mindset: testing, validation, and documentation are part of the job, not afterthoughts
Nice to have
- Databricks certification (certification support provided)
- Exposure to MLOps or feature engineering workflows
- Prior delivery experience in the GCC region
Interested candidates share their resume at [Confidential Information]
