Job Title: Data Platform Engineer – HBase / Amazon EMR / Spark

Number of Openings: 5

Experience: 7+ Years

Relevant Experience: 6+ Years

Employment Type: Full-Time

Work Mode: Remote

Shift Timing: 3:00 PM – 12:00 AM IST

Notice Period: Immediate Joiners or candidates serving notice up to 20 days

Job Summary

We are looking for experienced Data Platform Engineers with strong hands-on expertise in HBase, Amazon EMR, Apache Spark, Scala, and the Hadoop ecosystem. The ideal candidate will have proven experience managing production data platforms, migrating and modernizing HBase workloads, troubleshooting distributed systems, and ensuring data integrity during large-scale production migrations.

The role requires practical experience in HBase snapshots, HFiles, data movement, migration planning, production cutover, rollback procedures, and post-migration validation within AWS-based data platforms.

Key Responsibilities

• Design, manage, maintain, and troubleshoot production HBase and distributed data platforms.
• Execute HBase migration and modernization activities across clusters and environments.
• Work with HBase snapshots, Restore, ExportSnapshot, HFiles, and large-scale data movement techniques.
• Manage Amazon EMR clusters, including configuration, troubleshooting, upgrades, and workload migration.
• Develop, optimize, and troubleshoot Apache Spark applications using Scala and/or other relevant Spark programming.
• Work with Hadoop ecosystem components and understand dependencies between HBase, Spark, Hadoop, and application workloads.
• Integrate and manage data platforms using Amazon S3 and other AWS services.
• Plan and execute production migrations, cutovers, rollback strategies, and post-migration support.
• Validate migrated data using row counts, region counts, reconciliation, checksums, and other data integrity checks.
• Troubleshoot cluster-level issues related to performance, availability, connectivity, configuration, and compatibility.
• Ensure application stability and data consistency throughout migration and modernization activities.
• Develop automation scripts, operational runbooks, and repeatable migration and validation processes.
• Collaborate with Cloud, Architecture, Backend, DevSecOps, Security, and QA teams to ensure successful migration delivery.

Mandatory Skills and Experience

• 7+ years of overall IT experience, including 6+ years of relevant experience.
• Minimum 5 years of experience in Data Engineering or Big Data Engineering.
• Strong hands-on production experience with Apache HBase.
• Strong production experience with Amazon EMR.
• Hands-on experience with Apache Spark and Scala and/or Spark programming.
• Strong understanding of the Hadoop ecosystem and distributed data platforms.
• Experience working with AWS-based data platforms and Amazon S3.
• Proven experience in HBase migration, data platform modernization, or cluster migration.
• Practical knowledge of HBase snapshots, ExportSnapshot, HFiles, and restore procedures.
• Experience executing production data migration, cutover, rollback, and post-migration validation.
• Ability to validate data integrity using counts, reconciliation, checksums, or equivalent techniques.
• Strong troubleshooting skills involving distributed clusters and production workloads.
• Understanding of dependencies between HBase, Spark, Hadoop, and business applications.
• Experience in production support and migration-related issue resolution.

Good-to-Have Skills

• Experience with HBase 1.x/2.x and Amazon EMR 5.x/7.x.
• Knowledge of HBase Coprocessors and Apache Phoenix.
• Exposure to Hive, Presto, Apache Iceberg, Kerberos, and AWS KMS.
• Experience with encrypted data platforms and enterprise security requirements.
• Experience with large-scale enterprise data platform modernization.
• Strong documentation, automation, and cross-functional collaboration skills.

Candidate Profile

The ideal candidate should have strong analytical and problem-solving skills, hands-on production troubleshooting experience, and the ability to independently execute complex data migration activities. Candidates should be comfortable working in a remote environment during the specified shift.

Important Requirements

• Immediate joiners or candidates with a notice period of up to 20 days only.
• Must be comfortable working remotely from 3:00 PM to 12:00 AM IST.
• Candidates must have demonstrable hands-on experience in HBase, Amazon EMR, and HBase migration/modernization.

Interested candidates: Please share your updated resume along with total experience, relevant experience, current CTC, expected CTC, notice period, and details of your HBase migration experience.