Search by job, company or skills

s&p global mobility

Lead Data Engineer (AWS Cloud, Python, Distributed Data Processing)

Save
new job description bg glownew job description bg glow
  • Posted an hour ago
  • Be among the first 10 applicants
Early Applicant

Job Description

S&P Global has recently announced the intent to separate our Mobility Segment into a standalone public company.

For more information, visit www.spglobal.com/mobility.

The Role

We are seeking a Lead Data Engineer with extensive experience as a hands-on developer to join our fast-paced data engineering organization. This role is critical in building new data products and modernizing existing data platforms.

The ideal candidate is a seasoned data engineer with a strong background in designing, coding, and optimizing large-scale data pipelines, working deeply with Python, distributed data processing frameworks, and SQL across both on-premises and cloud environments. This individual will also provide technical leadership, guiding other engineers through complex data design and implementation challenges.

Responsibilities

Data Engineering & Architecture

  • Design, build, and maintain scalable, production-grade data pipelines and data products.
  • Develop and optimize ETL/ELT workflows using Python and distributed data processing technologies.
  • Implement and evolve data models that support both operational and analytical use cases.
  • Modernize legacy data platforms, including refactoring and migrating workloads to cloud environments.
  • Diagnose and resolve complex data issues related to performance, data quality, and pipeline reliability.

Cloud-Based Data Development

  • Build data solutions using a variety of cloud services for storage, processing, orchestration, and analytics.
  • Design efficient data processing architectures leveraging distributed computing frameworks.
  • Optimize data workloads for scalability, performance, and cost efficiency in the cloud.
  • Write and maintain infrastructure-as-code scripts to provision and manage data platform resources required for pipelines and jobs.

Technical Leadership & Mentoring

  • Lead by example through hands-on development, technical design, and code ownership.
  • Participate in and drive code and architecture reviews, ensuring high standards for quality, performance, and maintainability.
  • Mentor and support data engineers in Python, distributed data processing, SQL, and cloud-based data engineering practices.
  • Contribute to defining and evolving engineering standards and best practices.

Cross-functional Collaboration

  • Work closely with product managers, data analysts, software engineers, and other stakeholders to translate business requirements into well-designed data solutions.
  • Support and evolve the roadmap for data platform modernization and new data product delivery.

Required Qualifications

What We're Looking For:

  • 8+ years of experience in data engineering or a closely related technical role.
  • Strong background as a hands-on developer building data pipelines and data-intensive systems.
  • Advanced proficiency in Python and distributed data processing, including ETL/ELT development.
  • Strong expertise in SQL, data modeling, and query optimization.
  • Deep experience working with relational databases, including performance tuning and data extraction.
  • Hands-on experience developing data solutions in cloud environments.
  • Experience writing and maintaining infrastructure-as-code scripts for data platform infrastructure.
  • Solid understanding of distributed data processing concepts and architectures.

Preferred (Nice To Have)

  • Experience with modern cloud data warehouses.
  • Exposure to streaming or near–real-time data processing.
  • Familiarity with data governance, cataloging, or data quality tooling.

If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us!

It is the policy of Mobility to provide equal employment opportunity (EEO) to all persons regardless of age, color, national origin, citizenship status, physical or mental disability, race, religion, creed, gender, sex, sexual orientation, gender identity and/or expression, genetic information, marital status, status with regard to public assistance, veteran status, or any other characteristic protected by federal, state or local law. In addition, Mobility will provide reasonable accommodations for qualified individuals with disabilities.

More Info

Job Type:
Industry:
Employment Type:

About Company

Job ID: 147518733

Similar Jobs

Bengaluru, India

Skills:

composer NginxTomcatBigQueryShell ScriptingApacheApache AirflowGitDockerTerraformKubernetesPythonGitOpsPipelinesGithub ActionsGoogle Cloud ServicesOracle Weblogic DatabaseDAGsYAML files

Bengaluru, India

Skills:

amazon dynamodb S3LambdaTerraformSqsSnsMicrosoft Sql ServerEventBridge SchedulerAWS Serverless ArchitectureTerragrunt

Bengaluru, India

Skills:

snowflake AWS GlueInformaticaAzure Data FactoryGcpAzurePythonAWSNoSQL databasesETL frameworksdata lakesSQL databases

Bengaluru, India

Skills:

MatplotlibTableauExcel VbaSqlNumpyPandasSeabornAzurePythonAws S3MasterCard BrighterionMasterCard DIFalcon FMFico BlazeVRMRData visualization toolsSAS FM

Bengaluru, India

Skills:

NumpyPandasInformaticaSSISTalendPythonSqlEtl