About Position:
We are seeking a highly experienced Principal Data Architect (EMR) to lead large-scale data platform transformation initiatives, including migration from legacy Hadoop ecosystems to modern AWS-based data platforms.
This role requires a blend of deep technical expertise, architectural leadership, and delivery ownership, working closely with customer stakeholders, internal engineering teams, and AWS specialists to design, implement, and optimize enterprise-grade data platforms.
- Role: Principal Data Architect (EMR)
- Location: All PSL
- Experience: 15yr+
- Job Type: Full Time Employment
What You'll Do:
Architecture & Roadmap
- Define low-level architecture aligned with enterprise data platform (DAP) strategy
- Lead end-to-end data platform design on AWS EMR, including compute and service mapping
- Establish migration roadmap (domain waves, cutover strategies, legacy decommissioning)
- Design scalable, resilient EMR cluster architectures and multi-tenancy strategies
- Build disaster recovery and high availability frameworks
- Define data lineage architecture using AWS DataZone and Glue Catalog
- Recommend EMR versions, feature adoption, and modernization patterns
- Define metadata migration strategy from Hive Metastore to AWS Glue Catalog
- Provide guidance on on-prem to AWS cloud migration strategy
- Participate in architecture governance and review forums
Platform Implementation
- Drive implementation of target data platform on AWS, ensuring architectural alignment
- Define and standardize data, schema, and workflow conversion patterns
- Establish best practices for:
- Cluster sizing & instance selection
- Auto-scaling policies
- Cost optimization
- Lead Spark performance tuning strategy(execution plans, memory optimization, shuffle tuning)
- Build performance benchmarking and testing frameworks
- Support capacity planning and cost forecasting
- Collaborate with AWS Solution Architects and Specialist teams for escalations and optimization
Troubleshooting & Optimization
- Perform root cause analysis of platform and migration issues
- Resolve complex technical challenges in non-production and production environments
- Implement solutions to improve performance, reliability, and scalability
Delivery Leadership
- Provide technical leadership across multi-team programs, including less experienced architects and engineers
- Drive data migration, refactoring, and optimization efforts
- Establish and validate non-functional requirements:
- Reliability
- Performance
- Cost efficiency
- Observability
- Ensure go-live readiness through rigorous testing and governance checks
Expertise You'll Bring:
Technical Expertise
- 15+ years in Big Data and Data Platform Engineering
- Strong hands-on expertise with AWS EMR (EC2, EKS, Serverless)
- Deep experience in Hadoop ecosystem and large-scale migrations
- Strong knowledge of:
- Apache Spark (internals, execution plans, optimization)
- AWS Glue Data Catalog, Lake Formation, DataZone
- Proven experience in designing enterprise-scale data platforms and target architectures
- Familiarity with:
- Data governance frameworks
- Metadata management systems
- Data lineage tracking
Leadership & Consulting Skills
- Proven track record of leading large-scale, end-to-end migration programs
- Strong stakeholder management and ability to influence architecture decisions
- Experience working with cross-functional and distributed teams
- Excellent analytical, problem-solving, and decision-making skills
- Strong communication skills for both technical and business audiences
- Education: Bachelors or Master's degree in Computer Science, Engineering, or related field
Benefits:
- Competitive salary and benefits package
- Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
- Opportunity to work with cutting-edge technologies
- Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
- Annual health check-ups
- Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents
Values-Driven, People-Centric & Inclusive Work Environment:
Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.
- We support hybrid work and flexible hours to fit diverse lifestyles.
- Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
- If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment
Let's unleash your full potential at Persistent - persistent.com/careers
Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind.
