We are seeking a mid-level Big Data Developer to design, develop, and maintain data pipelines and analytical solutions using Spark Scala platform. The ideal candidate will have hands-on experience with Spark/Scala, strong data engineering skills, and the ability to collaborate with cross-functional teams to deliver business value.
Profile
- 4+ years of expertise and hands on experience in Spark with Scala and Big data technologies
- Good working experience in Scala and Spark.
- Good working experience in HDFS, Spark, Hive and Oozie
- Expertise in identifying bottlenecks and optimizing Spark jobs, HDFS storage, and Hive queries for maximum efficiency
- Technical expertise with data models, data mining, and partitioning techniques
- Hands-on experience with SQL database
- Have good understand on CI/CD tools (Maven, Git, Jenkins) and SONAR
- Good to have knowledge on Databricks
- Good to have knowledge on Kafka and ELK stack. Knowledge on Data visualization tool like Power BI will be an added advantage.
- Strong problem-solving and analytical skills
- Mentoring and leadership experience
- Skilled at working with cross-functional teams, translating business requirements into technical solutions, and presenting insights to stakeholders.
