Search by job, company or skills

hackajob

Lead Software Engineer - Apache Spark, AWS

hackajob
Bengaluru, India
5-7 Years
Save
  • Posted a day ago
  • Be among the first 10 applicants
Early Applicant

Job Description

hackajob is collaborating with J.P. Morgan to connect them with exceptional professionals for this role.

Job Description

We have an opportunity to impact your career and provide an adventure where you can push the limits of whatâs possible.

As a Lead Software Engineer at JPMorgan Chase within the Engineer Services Platform (ESP) in the Chief Technology Office (CTO), you are an integral part of an agile team that builds and operates a core capability in the firmâs artifact engineering ecosystem that enables secure, stable, and scalable consumption of open source software. As a core technical contributor, you will lead critical technology solutions that optimize open source software usage across the firm by strengthening governance controls and delivering actionable transparency through usage and compliance health metrics (per internal open source governance documentation).

Job responsibilities

  • Design, develop, and troubleshoot platform services that govern open source acquisition, usage, and compliance at scale
  • Design and implement scalable data platforms and pipelines (batch and streaming) with strong reliability and data quality controls.
  • Build event-driven/CDC-based architectures to enable near real-time data movement and downstream consumption patterns.
  • Develop and operate pipelines using technologies such as Spark, Kafka, and Airflow, focusing on maintainability and observability.
  • Design distributed data systems with strong understanding of partitioning/sharding, scalability, and performance optimization.
  • Build solutions on AWS and cloud-native data services; ensure security, governance, and cost-aware design.
  • Optimize end-to-end performance (ingestion, processing, storage, query), including SLAs and operational readiness.
  • Partner with stakeholders to define data contracts, integration patterns, and operational runbooks for production support.
  • Contribute to engineering excellence via design reviews, code reviews, and best practices for pipeline testing and operational stability.

Required Qualifications, Capabilities, And Skills

  • Formal training or certification on software engineering concepts and 5+ years applied experience.
  • Hands-on practical experience delivering system design, application development, testing, and operational stability for platform services used by many engineering teams.
  • Advanced in one or more programming language(s), with demonstrated ability to build production services and integrations used in critical developer workflows.
  • Proficiency in automation and continuous delivery methods, including designing and operating integrations that can enforce policy decisions in CI/CD pathways (per internal AMS integration expectations).
  • Proficient in all aspects of the Software Development Life Cycle, with strong engineering hygiene around secure coding, peer review, observability, and incident response.
  • Strong data engineering skills, including hands-on experience designing and building data pipelines and well-versed with the Medallion Architecture (Bronze, Silver, and Gold layers) to support reliable, scalable, and audit-ready data flows for compliance telemetry and open source health metrics.
  • Advanced understanding of agile methodologies and modern engineering practices such as CI/CD, application resiliency, and security, particularly as they apply to supply chain controls and governance automation.
  • Demonstrated proficiency in software applications and technical processes within a technical discipline such as cloud-native services, distributed systems, developer platforms, or security tooling.
  • In-depth knowledge of the financial services industry and its IT systems, including expectations for traceability, control alignment, and audit-ready evidence.
  • Practical cloud-native experience, including service design, deployment patterns, and runtime operations.

Preferred Qualifications, Capabilities, And Skills

  • Experience with NoSQL technologies such as Cassandra, MongoDB or DynamoDB.
  • Familiarity with big data technologies such as Spark, Scala, and HDFS.
  • Familiarity with modern infrastructure technology and architecture, including cloud, virtualization, and object stores.
  • Experience implementing or operating open source governance controls, including license risk classification approaches (e.g., Red/Yellow/Green models) and approval workflows for restricted licenses (per internal license governance guidance, including examples such as AGPL-3.0 and enforcement expectations).
  • Experience building compliance and posture dashboards that translate technical telemetry into operational health metrics and actionable insights for engineering and governance stakeholders.

ABOUT US

More Info

Job Type:
Permanent Job
Industry:
Other
Function:
Software Engineering
Employment Type:
Full time

About Company

hackajob

Job ID: 150843887

Jobs by Skill - IT
Jobs by Skill - Non IT
International Jobs
Last Updated: 07-07-2026 06:54:01 AM
Homejobs in Bengaluru / BangaloreLead Software Engineer - Apache Spark, AWS

Similar Jobs

Lead Software Engineer - Apache Spark, AWS

Jpmorgan & Co
Bengaluru, India

Skills:

data engineering , Distributed Systems, Spark, Kafka, AWS, Airflow, automation and continuous delivery methods, Medallion Architecture, cloud-native services

Lead Software Engineer - Apache Spark, AWS

JP Morgan Chase & Co.
Bengaluru

Skills:

Cassandra, automation, Dynamodb, Continuous Delivery, Kafka, Distributed Systems, AWS, MongoDB, Spark, Airflow, HDFS, data pipelines, Medallion Architecture

Security Engineer, AWS Security Incident Response

amazon web services (aws)
Bengaluru, India

Skills:

.NET, secure coding , threat modeling , Java, Aws Services, Software Development, Network security, Identity Management, System Administration, Incident Response, Authentication, Ruby, Swift, Cryptography, Python, Command line tools, Troubleshooting systems issues, Go, Security Operations, Analyzing logs, Risk management

AWS IoT

Tekskills India Private Limited
Hyderabad, Bengaluru, Chennai

Skills:

Aws, Mes

Early Applicant

SSE - Python + AWS + GenAI + Microservices/REST API

Quest Global
Bengaluru, India

Skills:

Microservices, Docker, Terraform, Distributed Systems, Kubernetes, Python, AWS, embeddings, Generative AI, vector databases, Agentic AI workflows, RAG pipelines, Large Language Models