Software QA Engineer- LLM Features

26WD94634

PositionOverview

Fusion isa cloud-based3D modeling, CAD, CAM, CAE, and PCB software platform for professionalproduct design and manufacturing. We're lookingfor amotivated SoftwareQA Engineer to jointhe Design Engineering QAteam, focusing on testingand validating Large Language Model(LLM) features integrated into Fusion. Thisrole will ensurethe quality, accuracy, and reliability of AI-powered conversational and generative capabilitieswithinthe product.

This is a hybrid position in Pune, India. You will report to Senior Manager, SQA.

Responsibilities

• Performmanual and automated testing forLLM-powered features toensure responsequality, accuracy, and consistency.

• Designand execute testcases for prompt-response validation, including edgecases and adversarial inputs.

• Identifyanddocument LLM-specificissuessuchas hallucinations, inaccurateresponses, and inconsistent outputs.

• Test LLM safetyand guardrails includingcontentmoderation, inappropriateresponsehandling, and prompt injectionprevention.

• Validate LLM integrationwithFusion workflowsand features, ensuringseamless userexperience.

• Trackand verifyLLM behaviorconsistencyacross modelupdates and versions.

• Develop automation scripts forLLM testing usingin-house toolsand frameworks.

• Collaboratewith development teams to ensuretimely defect resolution.

• Validateand reproducecustomer-reported LLM issues.

• Completeassigned testingtasks ontime and reportprogressregularly.

MinimumQualifications

• Bachelor's degree in ComputerScience, SoftwareEngineering, or related field.

• 2+ years of software QA experience.

• Basicunderstanding of LLM/AI concepts(prompts, tokens, modelresponses, contextwindows).

• Experience withtest casedevelopment and defect tracking.

• Familiarity with Pythonorscripting languages for test automation.

• Experiencewith bugtracking systems and testmanagementtools.

• Good analyticaland problem-solving skillswithattention to detail.

• Ability to workeffectivelyin cross-functional teams.

• Canworkindependently onday-to-day activities.

Preferred Qualifications

• Experience testing LLMorconversational AI applications(chatbots, AIassistants).

• Familiarity withLLM evaluation concepts(responserelevance, factual accuracy, coherence).

• Experiencewith promptengineering orprompttesting.

• Knowledgeof LLM safety testing(biasdetection, contentfiltering, jailbreakprevention).

• Hands-on experience with3D CAD/CAE software(Fusion 360, Inventor, etc.).

• Familiarity with APItesting tools(Postman, RESTclients).

• Familiarity withAgile/Scrum developmentmethodologies.

• Experience with AI/ML testingframeworks orevaluationtools.

#LI-CL1

Learn More

About Autodesk

Welcome to Autodesk! Amazing things are created every day with our software - from the greenest buildings and cleanest cars to the smartest factories and biggest hit movies. We help innovators turn their ideas into reality, transforming not only how things are made, but what can be made.

We take great pride in our culture here at Autodesk - it's at the core of everything we do. Our culture guides the way we work and treat each other, informs how we connect with customers and partners, and defines how we show up in the world.

When you're an Autodesker, you can do meaningful work that helps build a better world designed and made for all. Ready to shape the world and your future Join us!

Salary transparency

Salary is one part of Autodesk's competitive compensation package. Offers are based on the candidate's experience and geographic location. In addition to base salaries, our compensation package may include annual cash bonuses, commissions for sales roles, stock grants, and a comprehensive benefits package.

Diversity & Belonging
We take pride in cultivating a culture of belonging where everyone can thrive. Learn more here:

Are you an existing contractor or consultant with Autodesk

Please search for open jobs and apply internally (not on this external site).