Senior Data Engineer

We are hiring Senior FDE professionals with strong expertise in Databricks, Apache Spark, Data Engineering, Cloud, Data Migration and GenAI/RAG.

Required Skills :

• 10+ years overall experience with 7+ years in Data Engineering.
• Strong hands-on experience with Databricks.
• Strong expertise in Apache Spark, PySpark & Spark SQL.
• Hands-on experience with Data Migration / Cloud Migration / Data Modernization.
• Strong Data Architecture / Solution Architecture experience.
• Experience with AWS, Azure or GCP.
• Strong understanding of Delta Lake & Unity Catalog.
• Experience with batch & streaming data processing.
• Strong Python/PySpark coding skills.
• Experience with CI/CD and DevOps.
• Exposure to GenAI, LLM, RAG and MLOps/MLflow.
• Strong production troubleshooting, debugging and RCA skills.
• Customer-facing experience with strong stakeholder communication and technical solutioning.

Key Responsibilities :

• Design and implement scalable Databricks & Spark-based data solutions.
• Lead data migration and modernization initiatives.
• Design enterprise Lakehouse and data architectures.
• Develop production-grade data pipelines and integrations.
• Optimize Spark workloads and troubleshoot complex production issues.
• Implement batch and streaming data solutions.
• Work with Unity Catalog, Delta Lake and Databricks ecosystem.
• Contribute to GenAI/RAG and MLOps solutions.
• Engage with customers and stakeholders to understand requirements and provide technical solutions.
• Drive architecture discussions, technical design and end-to-end implementation.