Senior Hadoop Developer

Job description

• Work proactively to understand domain (Market Research) and technical trends that will impact/drive the development and design
• Work collaboratively/individually to write high quality, testable code that meets the Definition of Done
• Strive to meet team commitments
• Strive to eliminate technical debt
• Adopt a Test-First approach to software development, where possible in which the test is developed before the code
• Participate in design and code reviews with other engineers
• Use best software engineering practices to evolve the system design
• Perform unit/module testing of software
• Continuously evaluate software performance, identify performance degradation and opportunities to gain performance improvements valued by customers
• Responsible for assigned integration, testing and deployment tasks
• Take ownership of problems and continuously strive to become a better problem solver
• May establish and maintain software configuration control and software development environments including program source control to ensure reliable backups and source code integrity
• Write and maintain documentation to describe program development, logic, coding, testing, changes, and corrections
• Complex problem solving skills
• Strong analytical skills and the ability to understand complexities and how components connect and relate to each other
• Strong communication skills
• Strong teamwork skills for cross group cooperation
• Strong technical knowledge across the industry
• Leadership skills and ability
• Coaching and mentoring skills

Technical capabilities:

• Minimum 6 year hands on below skill sets
• Good hands and understanding in hive, HDFS file system and HDFS commands
• Good troubleshooting knowledge using Spark/Scala or HDFS file system
• Very good knowledge Hadoop architecture end to end
• Very good hands on in writing Spark SQL
• Very good hands on in writing Scala code for a given requirementVery good hands on writing Python scripts
• Very good hands on writing SQL/HQL scripts
• Very good hands on writing SQL/HQL scripts
• Good hands on Python libraries like pandas is preferred