Required Skills
- Must be proficient in the major languages such as Scala, Spark, and Kafka (any variant). Write elegant and maintainable code, and be comfortable with picking up new technologies.
- Proficient in working with distributed systems and have experience with different distributed processing frameworks that can handle data in batch and near real-time pipelines.
- Experience in working with AWS cloud services and Databricks to build end-to-end data lakehouse solutions that bring different systems together
- Capability to understand and implement metadata-based data processing framework and methodologies
- Experience in working with CICD process - Jenkins, & GIT Actions
- Experience in working with monitoring and control stack - Prometheus/Grafana
Key Job Responsibilities
- Drive the backbone of our organizational data platform by building robust pipelines that turn complex data into actionable insights using AWS and the Databricks platform
- Design scalable, maintainable data systems with strong observability
- Monitor, log, alert, and ensure system robustness
- Build efficient pipelines for structured and unstructured data
- Interface with Product engineering, regulatory, and legal teams as necessary to ensure Lakehouse's compliance adherence
- Write clean, efficient code that handles massive amounts of structured and unstructured data
Interested candidates can share their updated resume to
[Confidential Information] 
