Data Engineering

Job Description

The Data Engineer is responsible for large scale data ingestion and data modeling initiatives, defines raw to curated data conversion standards, establishes robust data validation frameworks, optimizes data operations performance, and ensures secure data access and classification.

Required Skills & Experience:-

Exp:-4 to 6 Years

Location:-Bengaluru or else Pan India

Strong experience leading data engineering teams in enterprise or large scale environments.
Deep expertise in data ingestion architectures, schema design, data modeling, and transformation pipelines.
Hands on experience designing data validation and quality frameworks.
Proven ability to optimize performance and reliability of data platforms.
Experience with data access control, classification, and governance in regulated or complex environments.
Strong communication, technical decision making, and stakeholder management skills.
Strong hands-on experience with Python, SQL, and ETL/ELT pipeline development.
Experience with Azure data services: blob storage, one lake, Azure Functions, Azure AI Search Vector Index, Cosmos DB, Azure Data Factory, Fabric Warehouse etc.
Working knowledge of Azure AI Search + Foundry Knowledge, Azure Event Grid, Azure Machine Learning, Azure ML Feature Store.
Experience with RAG data pipelines, including document ingestion, chunking, metadata tagging, embedding generation, vector indexing, and retrieval validation.
Experience with alarm/event data processing, network telemetry, logs, ticket data, and topology/inventory datasets.
Ability to parse semi-structured data from CSV, JSON, logs, SNMP trap payloads, and alarm text.
Understanding of network topology concepts: device, interface, link, LAG, site, region, upstream/downstream, service dependency.
Experience with data quality frameworks, schema validation, deduplication, and anomaly detection.
Familiarity with OpenSearch, DynamoDB, Redshift, PostgreSQL, or vector databases.
Good understanding of security, access control, encryption, and audit logging in Azure