Software Engineer_AI Tester_Bhubaneshwar

JobRequirement

• BE,B.Tech,M.Tech,MCAorequivalentdegreeinComputerScience(orrelatedfield).
• 3-5yearsofQAexperience,withatleast2yearsfocusedonAI/MLtesting(LLMs,RAG,AIagents).
• ProficiencyinArtificialIntelligence,MachineLearning,AIAgents,andLargeLanguageModels(LLMs).
• StrongknowledgeofQAmethodologies,testdesign,functional/non-functionaltesting,anddefectlifecyclemanagement.
• SolidunderstandingofLLMevaluation,hallucinationtypes,promptbehavior,responsescoring,andqualitymetrics.
• GoodunderstandingofRAGconceptsincludingvectordatabases,embeddings,andretrievalrelevance.
• CodingproficiencyinPythonorJava(bothpreferred).
• Hands-onexperienceinWeb/APIautomationusingframeworkslikePlaywright,Selenium,orRESTAssured.
• Experienceinwritingautomationscripts,maintainingtestframeworks,andintegratingtestsuitesintoCI/CDpipelines.
• AbilitytoanalyzelargesetsofAIoutputsforpatternsandsystemicissues.
• Excellentanalyticalreasoning,problem-solving,andcommunicationskills.

JobResponsibilitie

• TestandvalidateLLMoutputs,ensuringaccuracy,correctness,completeness,consistency,usability,andhallucinationanalysis
• EvaluateRAGsystems,includingretrievalaccuracy,documentrelevance,contextconstruction,andfullresponsegenerationflows
• TestAIagentsandautonomousworkflows,validatingdecision-making,taskexecution,anderrorhandling
• DesignandexecuteAI-specificteststrategies:datasetcreation,edge-casetesting,adversarialtesting,pattern-basedtesting,andregressionvalidation
• Developevaluationframeworks,scoringrubrics,andbenchmarkingmodelsforAIqualityassessment
• AnalyzelargevolumesofAI-generatedresponsestoidentifypatterns,rootcauses,andissueclusters,insteadofisolateddefects
• KnowledgeoftestingconversationalAI,workflows,oragent-basedsystems
• Exposuretovectorsearchtoolsandembeddingqualityanalysis
• Validatefixesusingnewexamplesfromthesamepatterncategory,ensuringtruemodelimprovement
• CollaboratecloselywithAI/MLengineers,QAteams,andproductmanagerstoimproveAIaccuracyandperformance
• ContributetocontinuousimprovementofAIQApractices,automation,tools,andevaluationdatasets