Big Data Engineer

Job Profile

We are looking for a Big Data Engineer who will work on building, and managing Big Data Pipelines for

us to deal with the huge structured data sets that we use as an input to accurately generate analytics at

scale for our valued Customers. The primary focus will be on choosing optimal solutions to use for these

purposes, then maintaining,implementing, and monitoring them. You will also be responsible for

integrating them with the architecture used across the company. Applicants must have a passion for

engineering with accuracy and efficiency, be highly motivated and organized, able to work as part of a

team, and also possess the ability to work independently with minimal supervision.

To be successful in this role, you should possess

Collaborate closely with Product Management and Engineering leadership to devise and build the

right solution.

Participate in Design discussions and brainstorming sessions to select, integrate, and maintain Big

Data tools and frameworks required to solve Big Data problems at scale.

Design and implement systems to cleanse, process, and analyze large data sets using distributed

processing tools like Akka and Spark.

Understanding and critically reviewing existing data pipelines, and coming up with ideas in

collaboration with Technical Leaders and Architects to improve upon current bottlenecks

Take initiatives, and show the drive to pick up new stuff proactively, and work

as a Senior Individual contributor on the multiple products and features we have.

7+ years of experience in developing highly scalable Big Data pipelines.

In-depth understanding of the Big Data ecosystem including processing frameworks like Spark,

Akka, Storm, and Hadoop, and the file types they deal with.

Experience with ETL and Data pipeline tools like Apache NiFi, Airflow etc.

Excellent coding skills in Java or Scala, including the understanding to apply appropriate Design

Patterns when required.

Experience with Git and build tools like Gradle/Maven/SBT.

Strong understanding of object-oriented design, data structures, algorithms, profiling, and

optimization.

Have elegant, readable, maintainable and extensible code style.

You are someone who would easily be able to

Work closely with the US and India engineering teams to help build the Java/Scala based data

pipelines

Lead the India engineering team in technical excellence and ownership of critical modules; own the

development of new modules and features

Troubleshoot live production server issues.

Handle client coordination and be able to work as a part of a team, be able to contribute

independently and drive the team to exceptional contributions with minimal team supervision

Follow Agile methodology, JIRA for work planning, issue management/tracking

Additional Project/Soft Skills:

Should be able to work independently with India & US based team members.

Strong verbal and written communication with ability to articulate problems and solutions over

phone and emails.

Strong sense of urgency, with a passion for accuracy and timeliness.

Ability to work calmly in high pressure situations and manage multiple projects/tasks.

Ability to work independently and possess superior skills in issue resolution.

Should have the passion to learn and implement, analyse and troubleshoot issues