AI/ML Engineer

Job Role: Senior Software Engineer

Location: Hyderabad

Experience : 5 Years

About the Role

We are seeking a Senior Software Engineer to design and build large-scale identity resolution and graph-based data systems. This role will focus on constructing and optimizing distributed data pipelines that power an identity graph used for entity resolution, cross-device matching, and data enrichment.

You will work at the intersection of big data processing, graph modeling, and distributed systems, building high-performance pipelines that process billions of records across cloud infrastructure.

This is a hands-on engineering role requiring deep experience in Spark-based distributed systems and cloud-native data platforms.

What You'll Do

Design and implement large-scale identity resolution pipelines using Spark

Develop high-performance data processing services in Scala, Python and Java

Build and maintain graph-based identity models and Senior Software Engineer Identity Resolution

About the Role

We are seeking a Senior Software Engineer to design and build large-scale identity resolution and graph-based data systems. This role will focus on constructing and optimizing distributed data pipelines that power an identity graph used for entity resolution, cross-device matching, and data enrichment.

You will work at the intersection of big data processing, graph modeling, and distributed systems, building high-performance pipelines that process billions of records across cloud infrastructure.

This is a hands-on engineering role requiring deep experience in Spark-based distributed systems and cloud-native data platforms.

What You'll Do

Design and implement large-scale identity resolution pipelines using Spark

Develop high-performance data processing services in Scala, Python and Java

Build and maintain graph-based identity models and entity resolution workflows

Optimize distributed workloads on Databricks

Architect scalable matching, clustering, and weak-edge detection algorithms

Improve observability and performance of data processing jobs

Collaborate with data scientists and product teams to operationalize graph models

Enforce best practices in data quality, lineage, and governance

Required Skills & Experience

Strong hands-on experience with Apache Spark

Proficiency in Scala, Python and Java

Experience deploying and managing workloads in AWS

Experience building and optimizing jobs in Databricks

Deep understanding of distributed systems and large-scale data processing

Experience designing scalable ETL/ELT pipelines

Strong SQL skills and understanding of data modeling

Familiarity with cloud-native architectures (AWS preferred)

Nice to Have

Experience building or working with identity graphs

Experience with identity resolution / entity resolution systems

Knowledge of graph algorithms (transitive closure, connected components, clustering, matching)

Experience working with large-scale Parquet datasets

Exposure to graph databases (e.g., Neo4j, Neptune) is a plus

Understanding of privacy-safe data linking techniques