Project Role : Custom Software Engineer

Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs.

Must have skills : Grafana

Good to have skills : Apache Kafka

Minimum 7.5 Year(s) Of Experience Is Required

Educational Qualification : 15 years full time education

Summary:

As a Custom Software Engineer, a typical day involves developing tailored software solutions by designing, coding, and improving various components within systems or applications. The role requires working with modern frameworks and following agile methodologies to ensure the delivery of scalable and efficient software that meets unique business requirements. Collaboration with cross-functional teams and continuous refinement of software elements to enhance performance and functionality are integral parts of the daily workflow.

Roles & Responsibilities:

Expected to be an SME, collaborate and manage the team to perform.
Responsible for team decisions.
Engage with multiple teams and contribute on key decisions.
Provide solutions to problems for their immediate team and across multiple teams.
Lead efforts to identify and implement process improvements that enhance team productivity and software quality.
Mentor junior team members to support their professional growth and integration within the team.
Coordinate with stakeholders to align software development activities with business goals and project timelines. Design and implement the observability layer for Akura across application, integration, event, agent, audit, and cloud runtime components.
Define logging, metrics, tracing, alerting, and dashboard standards for platform services.
Establish end-to-end correlation across Guidewire events, Kafka topics, backend services, agent decisions, ServiceNow actions, and APEX UI views.
Ensure all services expose health, readiness, latency, throughput, error, and dependency metrics.
Work with Solution Architect and DevOps Engineer to align observability with cloud deployment and operational readiness needs.
Configure and support OpenTelemetry instrumentation across backend microservices, Kafka consumers/producers, API services, agent services, and connectors.
Configure OpenTelemetry Collector pipelines for telemetry ingestion, enrichment, filtering, batching, memory limiting, masking, and routing.
Implement OTLP receivers and exporters as required by target observability platforms.
Use OpenTelemetry context propagation to preserve trace IDs and correlation IDs across services.
Support OpenTelemetry configuration governance and reusable collector patterns.
Instrument Spring Boot / FastAPI microservices with logs, traces, and metrics.
Capture API latency, request volume, error rate, dependency failures, and service health.
Define dashboard views for Guidewire Adapter Service, Incident Evaluation Service, ServiceNow Connector, Agent Orchestration Service, Audit Service, and APEX Backend APIs.
Ensure application logs include correlation ID, event ID, service name, environment, severity, and processing status.
Support troubleshooting of service-to-service and API gateway interactions.
Build dashboards and alerts for Kafka / Amazon MSK event processing.
Monitor topic throughput, consumer lag, failed events, retry counts, DLQ volume, partition health, and event processing latency.
Provide visibility into topics such as: guidewire.claim.event.created, incident.evaluation.requested, agent.incident.decision.created, incident.creation.requested, audit.event.recorded
Support replay and failure investigation by correlating Kafka offsets, event IDs, and audit entries.
Work with Backend/Kafka Engineers to define observability hooks for producers, consumers, and stream processing applications.
Implement observability for AI/agent workflows, including agent invocation count, agent latency, decision outcome, confidence distribution, failed tool calls, and human-in-loop routing.
Ensure agent decision flows are traceable from input event to final action.
Capture agent execution metadata, decision status, tool-call failures, and audit references.
Work with Agent Integration Engineer to expose metrics for incident triage, context enrichment, and recommendation workflows.
Build operational dashboards using CloudWatch, Grafana, or the client-approved observability tooling.
Create dashboard views for: Platform health, API health, Kafka processing, Agent decision flow, Incident creation status, Audit write success/failure, Deployment/runtime health, Business-unit-level event visibility
Work with platform and support teams to define alert thresholds and escalation views.
Support centralized observability and business-unit-specific dashboard patterns where required.
Define alerts for service failures, API latency, Kafka consumer lag, event processing failures, DLQ growth, audit failures, and agent action failures.
Support alert grouping, prioritization, and correlation to reduce operational noise.
Provide dashboards that help support users identify whether an issue is in Guidewire ingestion, Kafka processing, agent decisioning, ServiceNow integration, or APEX UI display.
Work with support and operations teams to define actionable alerts and runbook links.
Ensure telemetry does not expose sensitive data unnecessarily.
Implement masking, filtering, and routing controls for logs, metrics, and traces.
Support TLS/mTLS-based secure telemetry forwarding where applicable.
Work with Security/DevOps Engineer to align telemetry collection with security, privacy, and compliance requirements.
Support audit-friendly telemetry retention and access-control practices.
Define observability acceptance criteria for each platform component.
Create runbooks for alert investigation, service health checks, failed event triage, Kafka lag analysis, agent decision failure review, and dashboard usage.
Support production readiness reviews and operational handover.
Work with QA Engineer to validate observability data as part of end-to-end testing.
Work with DevOps Engineer to ensure observability deployment is automated and repeatable.

Professional & Technical Skills:

Must To Have Skills: Proficiency in Grafana.
Good To Have Skills: Experience with Apache Kafka.
Strong knowledge of software development lifecycle and agile methodologies.
Ability to design and implement scalable software components.
Experience in monitoring and visualizing system performance using relevant tools.
Familiarity with integrating data streams and event-driven architectures.
Hands-on experience with OpenTelemetry instrumentation and collector configuration.
Experience with CloudWatch, Grafana, or equivalent observability platforms.
Strong understanding of metrics, logs, traces, dashboards, and alerting.
Experience instrumenting microservices, APIs, and distributed systems.
Understanding of Kafka/event-driven monitoring concepts such as consumer lag, topic throughput, failed events, retries, and DLQs.
Experience working with containerized workloads on Kubernetes, EKS, ECS, or equivalent runtime.
Ability to define correlation ID and trace propagation standards.
Experience creating production dashboards and alerting rules.
Good understanding of operational troubleshooting and production support.
Ability to collaborate with developers, DevOps, QA, security, and architecture teams.
Amazon MSK / Apache Kafka monitoring experience.
Experience with Prometheus, Loki, Tempo, Jaeger, Dynatrace, Splunk, Azure Monitor, or Application Insights.
OpenTelemetry Transformation Language / OTTL exposure.
Experience with telemetry masking, filtering, enrichment, and routing.
Experience with Guidewire Cloud or insurance platform telemetry.
Experience with ServiceNow incident integration.
Experience with AI/agent workflow monitoring.
Experience with GitHub Actions, Terraform, Helm, and GitOps-style observability deployment.
Experience supporting SRE-style operational dashboards and service-level indicators.

Additional Information:

The candidate should have minimum 7.5 years of experience in Grafana.
This position is based at our Bengaluru office.
A 15 years full time education is required.