Scala Developer

Roles & Responsibilities

Design & implement robust, scalable, batch & real-time data engineering solutions using Apache Spark (Scala) & Spark structure streaming.

• Architect well-structured Scala projects using reusable, modular, and testable codebases aligned with SOLID principles and clean architecture principles & practices.

• Develop, Deploy & Manage Spark jobs on Kubernetes clusters, ensuring e icient resource utilization, fault tolerance, and scalability. • Orchestrate data workflows using Apache Airflow — manage DAGs, task dependencies, retries, and SLA alerts.

• Write and maintain comprehensive unit tests and integration tests for Pipelines / Utilities developed.

• Work on performance tuning, partitioning strategies, and data quality validation.

• Use and enforce version control best practices (branching, PRs, code review) and continuous integration (CI/CD) for automated testing and deployment.

• Write clear, maintainable documentation (README, inline docs, docstrings).

• Participate in design reviews and provide technical guidance to peers and junior engineers.

Mandatory Skills

SCALA , Spark, Data Engineering, Pyspark