About Position:

We are looking for an experienced Big Data Software Engineer to join a highly skilled technology team focused on innovation, learning, and prototyping next-generation data solutions. As part of the Ingestion Services team, you will contribute to building and standardizing enterprise data ingestion frameworks using modern Big Data technologies.

Role: Senior Big Data Software Engineer - Spark, Scala & Databricks
Location: All Persistent Locations
Experience: 5 to 10 Years
Job Type: Full-Time Employment

What You'll Do:

Understand business requirements and align technical solutions with enterprise architecture and global solution design.
Design, develop, and enhance Big Data ingestion solutions using Spark, Scala, Databricks, and PySpark.
Configure and implement application and integration services to support business and platform requirements.
Build scalable and reusable data processing frameworks for enterprise data ingestion and transformation.
Develop proof of concepts, prototypes, and innovative solutions aligned with business strategies.
Follow source-driven development, test-driven development, and engineering best practices.
Design, build, and maintain CI/CD pipelines using GitHub and GitHub Actions.
Develop and execute unit tests and integration tests to ensure solution quality and reliability.
Troubleshoot production issues, perform root cause analysis, and implement effective resolutions.
Collaborate with architects, developers, analysts, and business stakeholders across global Agile/Scrum teams.
Support production environments and continuously improve platform performance, reliability, and scalability.
Contribute to DevSecOps initiatives and secure software engineering practices.
Participate in Agile ceremonies including sprint planning, daily stand-ups, code reviews, retrospectives, and release activities.

Expertise You'll Bring:

5-10 years of experience in Big Data Engineering and enterprise software development.
Strong hands-on expertise in Apache Spark, Scala, and Databricks.
Proven experience developing large-scale data ingestion and processing solutions using PySpark.
Strong understanding of distributed computing and Big Data processing frameworks.
Experience designing and implementing scalable, standardized data ingestion architectures.
Strong SQL skills including query writing, data analysis, and database functions.
Hands-on experience using GitHub for source code management and collaborative development.
Experience building and supporting CI/CD pipelines using GitHub Actions.
Knowledge of Linux/Unix environments, shell scripting, and command-line tools.
Experience with unit testing, integration testing, test automation, and test-driven development practices.
Understanding of Agile/Scrum methodologies and working within globally distributed teams.
Exposure to cloud-native and cloud-agnostic engineering approaches.
Working knowledge of Azure cloud services and modern data engineering ecosystems.
Awareness of AI-enabled engineering practices, AI readiness initiatives, and DevSecOps methodologies.
Strong troubleshooting, debugging, analytical, and problem-solving skills.
Excellent communication, collaboration, and stakeholder management capabilities.

Benefits:

Competitive salary and benefits package
Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
Opportunity to work with cutting-edge technologies
Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
Annual health check-ups
Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

We support hybrid work and flexible hours to fit diverse lifestyles.
Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment

Let's unleash your full potential at Persistent - persistent.com/careers

Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind.

Senior Big Data Software Engineer