Data Engineer (Spark/Streaming)

5-10 years
2 months ago 308 Applied
Job Description

Job Role : Data Engineer

Experience : Spark / Streaming 2-3 Years 7-9 Years
Location : Pune/Bangalore/Chennai/Hyderabad/Mumbai
Notice Period : max 30 Days
Skills : Big Data (Hadoop, Hive, Hbase) + SQL
Must Have Skills (As per TAC) : Data Engineer (Spark/Streaming)

Roles & Responsibilities :
1. Develop and maintain architectures such as databases and large-scale processing systems using Python, Scala, SQL, and Unix scripting.
2. Utilize Big Data technologies, including Hadoop, Hive, Hbase, and Spark to manage and manipulate large data sets.
3. Leverage your recent experience with Spark Streaming in project deployments.
4. Write and optimize SQL and HQL for database management and data manipulation.
5. Implement coding best practices to maintain software quality and readability.
6. Optimize queries and tune performance for Hive and Spark applications.
7. Collaborate with team members to design, develop, and implement robust data solutions.
8. Stay abreast of the latest industry trends and technologies.

Good to have / desirable Skills :
1. A minimum of 7-9 years of industry experience in data engineering.
2. Strong programming skills with at least 7 years of experience in Python, Scala, and SQL.
3. Expertise in Big Data technologies including Hadoop, Hive, Hbase, and Spark.
4. Proficiency in Unix scripting with a minimum of 7 years of experience.
5. Recent experience (2-3 years) with Spark Streaming on a project.
6. Deep knowledge of SQL and HQL.
7. Solid understanding of coding best practices.
8. Experience in query optimization and performance tuning, particularly for Hive and Spark.
9. Excellent problem-solving skills and attention to detail. 10. Strong communication and teamwork skills.

Competency for the roles :
1. Big Data Expertise: The candidate should have a strong understanding of Big Data technologies such as Hadoop, Hive, Hbase, and Spark. They should be capable of managing and manipulating large datasets using these technologies.
2. Spark Streaming Experience: The candidate should have recent and significant experience with Spark Streaming. This shows their ability to work with real-time data and perform stream processing.
3. Database Management: Proficiency in SQL and HQL is required. The candidate should be capable of creating, managing, and manipulating databases, as well as optimizing and tuning queries.


About

At Synechron, we believe in the power of digital to transform businesses for the better. Our global consulting firm combines creativity and innovative technology to deliver industry-leading digital solutions. Synechron’s progressive technologies and optimization strategies span end-to-end Artificial Intelligence, Consulting, Digital, Cloud & DevOps, Data, and Software Engineering, servicing an array of noteworthy financial services and technology firms. Through research and development initiatives in our FinLabs, we develop solutions for modernization, from Artificial Intelligence and Blockchain to Data Science models, Digital Underwriting, mobile-first applications, and more. Over the last 20+ years, our company has been honored with multiple employer awards, recognizing our commitment to our talented teams. With top clients to boast about, Synechron has a global workforce of 14,500+, and has 48 offices in 19 countries within key global markets.

People Also Considered

Career Advice to Find Better