Experience: 10+ Years
Job Type: Full-time
About Katonic:We are a young, Aussie Generative AI startup headquartered in Sydney, Australia.
Job Description:
We are seeking a highly experienced Senior QA Lead / QA Manager with 10+ years in quality engineering to own and drive the QA strategy for our enterprise-grade GenAI and MLOps platform. This platform includes LLM-based chatbots, knowledge search and retrieval, web search integration, text and document processing, image processing, connector integrations, persona and brand voice configurations, multi-tenant operations, model fine-tuning, GPU-based LLM deployments, and API-based chatbot interfaces. The ideal candidate will lead QA teams, define scalable test processes, and ensure end-to-end reliability, performance, and security across all AI/ML and platform components.
Key Responsibilities:
- Lead and manage the QA team, establishing QA strategy, processes, and governance.
- Design comprehensive test plans for GenAI features, including enterprise chatbot, RAG search, Ask-AI modules, guardrails, persona/brand voice workflows, and multimodal pipelines.
- Validate ML workflows such as embedding generation, model fine-tuning, LLM deployments on GPU, GPU slicing, resource allocation, and model lifecycle management.
- Test platform components: tenancy management, connectors, integrations, microservices, and multi-tenant SaaS modules.
- Build and maintain automation for UI, API, backend services, and ML pipelines (functional, regression, integration, and E2E).
- Perform performance, scalability, and load testing for chat-based and API-driven systems.
- Validate Node.js, Streamlit, Gradio, and Dash applications for stability and compatibility.
- Ensure quality coverage for document, text, image, and tabular data processing features.
- Collaborate with Engineering, AI/ML, and DevOps teams to ensure high-quality releases.
- Drive root-cause analysis, defect management, and continuous improvements across the QA lifecycle.
Required Skills:
- 10+ years of QA experience with at least 35 years in a lead or managerial role.
- Strong experience testing GenAI/ML systems, including LLMs, embeddings, RAG pipelines, and multimodal processing.
- Hands-on automation expertise using Selenium, Playwright, Cypress, Pytest, Robot Framework, or similar.
- Strong API testing experience (Postman, RestAssured, Playwright API, JMeter).
- Knowledge of Python, Node.js, or other scripting languages for test automation.
- Familiarity with GPU-based model deployment, Docker, Kubernetes, and cloud platforms.
- Experience testing multi-tenant SaaS applications, microservices, and real-time chat APIs.
- Strong analytical and debugging skills, with an ability to work closely with cross-functional teams.
Preferred Skills:
- Experience with MLflow, Kubeflow, or other MLOps tools.
- Knowledge of vector databases (FAISS, Pinecone, Chroma, Weaviate).
- Experience with guardrails frameworks (NeMo Guardrails, Guardrails AI, etc.).
- Exposure to GPU orchestration, vLLM, Triton, or Hugging Face model servers.
What We Offer:
- Opportunity to lead QA for a cutting-edge GenAI & MLOps ecosystem.
- Work with advanced LLMs, GPUs, embeddings, and multimodal AI technologies.
- Collaborative environment with engineering, AI/ML research, DevOps, and product teams.
- 100% work from home
Please apply only if you match the criteria.
To apply, please fill out this form here:https://shorturl.at/Of9gU
Without filling out the form, your application is not complete.
