Senior Analyst - AI QA (R-18975)

Shape the Future with Dun & Bradstreet

At Dun & Bradstreet, we believe data has the power to create a better tomorrow. As a global leader in business decisioning data and analytics, we help companies worldwide grow, manage risk, and innovate. For over 180 years, businesses have trusted us to turn uncertainty into opportunity. We're a diverse, global team that values creativity, collaboration, and bold ideas. Are you ready to make an impact and help shape what's next Join us! Explore opportunities at dnb.com/careers.

We are looking for a highly skilled AI Tool/Agent Testing Engineer to evaluate, validate, and ensure the reliability of AI agents, AI automation tools, and agentic workflows used across our analytics platform. This role blends test engineering, GenAI understanding, Python/PySpark proficiency, and agent development lifecycle knowledge. You will work closely with Data Science, AI Engineering, and Platform teams to ensure that AI agents behave predictably, safely, and in alignment with business and compliance requirements

• Build agentic workflows using Lang Chain/Lang Graph and similar frameworks


• Develop autonomous agents for data validation, reporting, document processing, and domain workflows


• Deploy scalable, resilient agent pipelines with monitoring and evaluation


• Develop GenAI applications using models like GPT, Gemini, and LLaMA


• Implement RAG, vector search, prompt orchestration, and model evaluation


• Partner with data scientists to productionize POCs


• Build distributed data pipelines (Python, PySpark)


• Develop APIs, SDKs, and integration layers for AI-powered applications


• Optimize systems for performance and scalability across cloud/hybrid environments


• Contribute to CI/CD workflows for AI modelsdeployment, testing, monitoring


• Implement governance, guardrails, and reusable GenAI frameworks


• Work with analytics, product, and engineering teams to define and deliver AI solutions


• Participate in architecture reviews and iterative development cycles


• Support knowledge sharing and internal GenAI capability building

• 6-8 years of experience in AI/ML engineering, data science, or software engineering, with at least 2 years focused on GenAI


• Strong programming expertise in Python, distributed computing using PySpark, and API development


• Hands on experience with LLM frameworks (Lang Chain, Lang Graph, Transformers, OpenAI/Vertex/Bedrock SDKs)


• Experience developing AI agents, retrieval pipelines, tool calling structures, or autonomous task orchestration


• Solid understanding of GenAI concepts: prompting, embeddings, RAG, evaluation metrics, hallucination identification, model selection, fine tuning, context engineering


• Experience with cloud platforms (Azure/AWS/GCP), containerization (Docker), and CI/CD pipelines for ML/AI


• Strong problem solving, system design thinking, and ability to translate business needs into scalable AI solutions


• Excellent verbal, written communication and presentation skills

All Dun & Bradstreet job postings can be found at https://jobs.lever.co/dnb. Official communication from Dun & Bradstreet will come from an email address ending in @dnb.com.

Notice to Applicants: Please be advised that this job posting page is hosted and powered by Lever, a subsidiary of Employ Inc. Your use of this page is subject to Employ's Privacy Notice and Cookie Policy, which governs the processing of visitor data on this platform.

We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please visit https://bit.ly/3LMn4CQ.