Search by job, company or skills

MindBrain

Big data engineer- Spark Scala- Visa

Early Applicant
  • 5 months ago
  • Be among the first 50 applicants

Job Description

Job Title: Big Data Engineer - Spark/Scala - Visa

Location: Bengaluru, Karnataka, India

Job Type: Full-time

Experience: 6+ years

---

Job Description:

Key Responsibilities:

- Design, develop, and maintain big data processing pipelines using Apache Spark and Scala.

- Optimize and enhance the performance of existing big data solutions.

- Work closely with data scientists, analysts, and other stakeholders to understand data requirements and deliver efficient solutions.

- Develop and maintain scalable data architectures and frameworks.

- Implement data processing workflows for batch and real-time data processing.

- Ensure data quality, integrity, and security across all data processing pipelines.

- Troubleshoot and resolve any issues related to data processing and analytics.

- Document technical solutions and maintain comprehensive documentation.

Required Qualifications:

- Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.

- 6+ years of experience in big data engineering.

- Extensive experience with Apache Spark and Scala.

- Strong understanding of big data technologies such as Hadoop, Hive, HBase, and Kafka.

- Proficient in SQL and experience with relational databases.

- Experience with cloud platforms like AWS, Azure, or Google Cloud.

- Strong problem-solving skills and the ability to troubleshoot complex issues.

- Excellent communication and collaboration skills.

- Experience with version control systems like Git.

Preferred Qualifications:

- Experience with data visualization tools such as Tableau, Power BI, or similar.

- Knowledge of machine learning frameworks and libraries.

- Familiarity with containerization and orchestration tools like Docker and Kubernetes.

- Experience with Agile methodologies and DevOps practices.

More Info

Industry:Other

Function:technology

Job Type:Permanent Job

Date Posted: 20/06/2024

Job ID: 82394907

Report Job

About Company

Follow

Hi , want to stand out? Get your resume crafted by experts.

Similar Jobs

Lead Data Engineer Full Stack Java Scala Big Data Spark Kafka Streaming 14 yrs

VisaCompany Name Confidential

Data Engineer Senior Lead Scala Spark Streaming Data pipeline German MNC

Valuable Consulting and Technologies Private LtdCompany Name Confidential
Last Updated: 20-06-2024 00:46:05 PM
Home Jobs in Bengaluru / Bangalore Big data engineer- Spark Scala- Visa