Lead Data Engineer

Job Description

Lead Software Engineer / Associate Architect with proficiency in design and implementation of scalable and robust data engineering solutions

Roles & Responsibilities

In-depth understanding of Big data concepts and distributed systems

Proficiency in Python and writing code in PySpark

Should possess good understanding of SQL concepts and must be proficient writing complex SQL queries

Thorough understanding of optimization and performance improvement for Spark based Workloads

Experience with working on atleast one Cloud platform will be a good to have

Conceptual knowledge and fitment for Data Structures & Algorithms

Possessing in-depth knowledge of various Data Processing Patterns (Batch/NRT/RT processing) & capable of providing design & architecture of typical business problems

Knowledge and experience with NoSQL Database (Cassandra/HBase/MongoDB/CouchDB/Neo4j),SQL Database (MySQL/Oracle).

Kafka, Redis, Distributed Message Queues alongwith Distributed Caching

Proficient understanding of Build tools (Maven/Gradle), Code Versioning tools (Git) with Continuous Integration

Strong technical development experience with writing performant code leveraging best coding practices.

Good communication skills and aptitude

Mandatory Skills: Bigdata, Pyspark , Python/Java

Years Of Experience: 8 to 12 Years