Search by job, company or skills

Early Applicant
Quick Apply
  • Posted 10 hours ago
  • Be among the first 20 applicants

Job Description

Job Title: Senior Data Engineer (Agentic Data Infrastructure)

Experience Range: 6+ Years (with strong focus on modern cloud data architectures)

Location: Bangalore / Hyderabad / Gurgaon/ Chennai (Hybrid)

Job Overview

We are seeking a highly skilled Senior Data Engineer to architect and build the foundational data infrastructure that powers enterprise-grade Generative AI and autonomous agent systems. Rather than simply building traditional batch-oriented data warehouses, you will be designing real-time, federated data substrates that allow AI agents to securely access, reason over, and orchestrate complex enterprise data in milliseconds.

In this role, you will solve systemic data fragmentation problems by building zero-copy architectures, low-latency event streams, and dynamic access layers across heterogeneous enterprise platforms. You will work closely with AI Architects to ensure data structures (whether relational, graph, or vector) are optimized specifically for LLM tool-calling and agentic reasoning under strict regulatory constraints.

Key Responsibilities

Agent-Ready Data Pipelines: Design and deploy low-latency, event-driven data pipelines (GCP Pub/Sub, Dataflow) that feed real-time context to stateless AI agents, shifting away from traditional nightly batch ETLs.

Federated Data Architecture: Build read-only integration layers and API wrappers that allow AI models to query disparate operational systems (legacy ledgers, risk engines, CRMs) in parallel without massive data duplication.

Optimized Data Modeling for AI: Design sophisticated BigQuery schemas, Spanner/AlloyDB relational models, and potentially Graph or Vector structures specifically tuned for high concurrency, low-latency LLM tool-calling and context injection.

Identity Resolution & Data Joining: Solve complex cross-system identity mapping challenges, ensuring AI agents have a unified, accurate 360-degree view of entities across fragmented enterprise brands and databases.

Governance & Pipeline Security: Implement automated data quality checks, PII redaction layers (Cloud DLP), and Attribute-Based Access Controls (ABAC) to ensure agents never ingest or expose restricted regulatory data.

Pipeline Orchestration & Cost Tuning: Orchestrate complex DAGs using Apache Airflow (Cloud Composer) while aggressively monitoring and optimizing compute resources (minimizing shuffle in Dataflow/Spark) to control cloud costs at enterprise scale. Required Qualifications

Data Engineering Experience: 6+ years of hands-on experience building robust, scalable data pipelines and modern data warehouses in enterprise environments.

Google Cloud Platform (GCP) Mastery: Strong operational experience with GCP data services, specifically BigQuery, Dataflow (Apache Beam), Pub/Sub, and Cloud Storage.

Real-Time & Event-Driven Systems: Proven ability to design streaming architectures and asynchronous data feeds, moving beyond traditional batch processing.

Advanced SQL & Data Modeling: Expert-level SQL optimization (joins, window functions) and experience designing schemas for both analytical (OLAP) and transactional/operational (OLTP) access patterns.

Pipeline Orchestration: Hands-on experience with Apache Airflow (Cloud Composer) or similar modern orchestration tools.

Distributed Frameworks: Working knowledge of distributed processing engines like Spark or Hadoop for heavy lifting when required.

Cross-Functional Problem Solving: Ability to tackle ambiguous data problems (e.g., identity resolution, federated querying) across siloed enterprise teams.

What Makes You Stand Out

Experience building data layers specifically designed for consumption by LLMs or autonomous agents (e.g., semantic layers, feature stores for AI).

Familiarity with low-latency operational databases on GCP (Cloud Spanner, AlloyDB, or Memorystore/Redis).

Experience in heavily regulated industries (FinTech, Banking, Healthcare) handling sensitive PII/financial data with strict compliance mandates.

Understanding of API integration and how to expose data assets as tools for Model Context Protocol (MCP) or similar agent frameworks.

Interview process:

1. Interview Process: MS Team (3 rounds.)

2. Working Location: Bengaluru, Hyderabad, Chennai, Gurugram

3. Work Mode: Hybrid (3 days in office, 2 days remote)

4. Notice Period: Immediate or max 15 Days

5. Shift Time: Based on Project

6. Experience: 6+ years

Bachelor Of Technology (B.Tech/B.E), Master of Accounting, Doctor of Medicine (M.D/M.S), Masters in Technology (M.Tech/M.E), Bachelor of Dental Surgery (B.D.S), Bachelor of Homeopathic Medicine and Surgery (BHMS), Masters in Arts (M.A), Master of Planning (M.Plan), Bachelor of Science (B.Sc), Bachelor in General Studies (BGS), Bachelors of Law (B.L/L.L.B), Bachelor of Applied Science (BAS), Master OF Business Administration (M.B.A), Bachelor of Ayurvedic Medicine and Surgery (BAMS), Bachelor of Arts (B.A), Master of Commerce (M.Com), Master of Law (M.L/L.L.M), Doctor of Audiology (AuD), Master of Education (M.Ed), Bachelor of Commerce (B.Com), Master of Liberal Arts (MLA/ALM), Doctor of Jurisprudence, Master of Music (MM/MMus), Bachelor of Business Administration (B.B.A), Bachelor Of Pharmacy (B.Pharm), Doctor of Ministry, Masters in Communications Disorders (MCD), Master of Fine Arts (MFA), Bachelor of Music Education (B.M.E.), Doctor of Psychology (Psy. D./D. Psych.), Master of Professional Accountancy (MPA), Bachelor of Social Work (B.S.W), MBBS, Doctor of Public Health (DrPH), Doctor of Musical Arts (DMA), Master of Tax Accounting, Master of Health Administration (MHA), Doctor of Science (DSc/ScD), Doctor of Veterinary Medicine (DVM), Bachelor of Music, Master of Health Science (MHSc), Master of Public Administration (MPA), Master of Real Estate Development, Master of Public Health (MPH), Bachelor of Education (B.Ed), Master of Pharmacy (M.Pharm), Master of Divinity, Masters of Architecture (M.Arch), Doctor of Education, Bachelor of Interdisciplinary Studies, Master of Natural Resources (MNR), Master of Library & Information Science (MLIS), Bachelor in Hotel Management (B.H.M), Bachelor of Architecture (B.Arch), Master of Social Work (M.S.W), Master in Landscape Architecture, Doctor of Physical Therapy, Doctor of Dental Surgery/Medicine (DDS/DMD), Doctor of Optometry, Doctor of Pharmacy (PharmD), Master in Computer Application (M.C.A), Master of Library Science, Doctor of Nursing Practice (DNP), Doctor of Chiropractic (DC), Master of Technical & Professional Communication, Bachelor of Health Science, Bachelor Of Computer Application (B.C.A), Doctor of Business Administration (DBA), Bachelor of Liberal Arts (BLA/ALB)

More Info

Job Type:
Function:
Employment Type:

Key Skills

Similar Jobs

5-7 yrs
Bengaluru, India
Skills:
data engineering , snowflake , BigQuery, Data Modelling, Kafka, Redshift, Sql, ELT, Hive, Kinesis, Presto, Docker, Terraform, Spark, Python, AWS, Etl, Airflow, Pubsub, Trino, Delta Lake
6-9 yrs
Bengaluru, India
Skills:
Azure Data Factory, Scala, Azure Synapse Analytics, Apache Spark, Microsoft Azure, Azure Databricks, Sql, Azure Event Hubs
6-12 yrs
Bengaluru, India
Skills:
Netsuite, Databricks, Data Integration, Etl, ELT, Salesforce
5-7 yrs
Bengaluru, India
Skills:
Measurement-driven optimization, Reading existing systems, Operations and reliability, SQL and relational data modeling
6-8 yrs
Bengaluru, India
Skills:
Elk, Pyspark, PostgreSQL, Prometheus, Kafka, Grafana, Terraform, Oracle, Python, AWS, Workflows, SQL Server, Sql, Git, Gcp, Spark, Databricks, Azure, GenAI, Infrastructure as Code, MLflow, CI CD, Unity Catalog, Delta Lake, Databricks Genie