Big Data Developer (Scala & Spark)

Position Description

We are looking for an experienced Big Data Developer with strong expertise in Scala and ApacheSpark to design, develop, and optimize large-scale data processing solutions. The role involves working on distributed data platforms, building scalable pipelines, and collaborating with cross-functional teams to deliver high-performance data-driven solutions.

Location

Bangalore / Chennai / Hyderabad / Pune

Required Qualifications

• Overall experience of 6+ years in Big Data application development
• Self-assessed proficiency in Scala and ApacheSpark for large-scale data processing
• Strong experience with Hadoop ecosystem components including HDFS and Hive
• Working knowledge of Unix/Linux environments
• Experience with DevOps practices including Jenkins CI, Git, and CI/CD pipelines
• Strong problem-solving, analytical, and communication skills.

Future Duties & Responsibilities

• Design, develop, and maintain scalable Big Data applications using Scala and Spark
• Build and optimize batch and streaming data pipelines within the Hadoop ecosystem
• Collaborate with data engineers, architects, and business stakeholders to deliver data solutions
• Implement CI/CD pipelines and follow DevOps best practices for deployment and monitoring
• Ensure code quality, performance optimization, and adherence to CGI standards
• Participate in continuous learning and contribute to technical improvements

Must Have Skills

• Scala with Apache Spark
• Big Data application development
• Hadoop ecosystem (HDFS, Hive)
• Unix/Linux
• DevOps tools: Jenkins CI, Git, CI/CD pipelines

Good to Have Skills

• Cloud platforms: AWS and/or Azure
• Databricks
• Python and PySpark
• Java with Spring / Spring Boot