Job Description

Spark/Scala Developer

We are seeking a skilled Spark/Scala Developer to join our innovative team in building and enhancing data processing pipelines. The ideal candidate will leverage their expertise in big data technologies to design, develop, and maintain scalable and efficient data processing solutions. This role is critical for ensuring our data infrastructure can handle large volumes of data with high performance and reliability. The right person will be a key player in our data-driven strategies and will work closely with cross-functional teams to deliver impactful results.

Key Responsibilities:

• Develop and maintain complex data processing pipelines using Spark and Scala.
• Optimize data processing workflows for speed and efficiency.
• Collaborate with data engineers to design robust data architectures.
• Implement scalable solutions for big data analytics.
• Ensure data integrity and accuracy across various data sources.
• Troubleshoot and resolve issues in data processing systems.
• Participate in code reviews and contribute to best practices in data engineering.
• Work with machine learning models to enhance data processing capabilities.
• Stay updated with the latest advancements in big data technologies.
• Support the deployment and maintenance of data processing systems.

Qualification Criteria:

• Proven experience as a Spark/Scala Developer.
• Strong understanding of big data technologies and frameworks.
• Proficiency in writing efficient and scalable code.
• Experience with data warehousing and database management.
• Excellent problem-solving skills and attention to detail.
• Ability to work in a fast-paced, collaborative environment.
• Strong communication skills to effectively liaise with cross-functional teams.
• Familiarity with cloud platforms such as AWS, Azure, or Google Cloud.
• Knowledge of data visualization tools and techniques.
• Experience with CI/CD pipelines and automated deployment processes.

Primary Skills:

• Spark
• Scala
• big data

Key Skills:

• Java
• Python
• SQL
• Hadoop
• Kafka
• Docker
• Kubernetes
• AWS
• Azure
• Google Cloud
• ETL
• data engineering
• data warehousing
• data visualization
• CI/CD