Office of Small Failures / Case File 006AI brand accuracy
AI answer audit experiment

AI Accuracy Audit Tool: AI Said What?

Define the facts that matter, paste an answer from ChatGPT, Gemini, Perplexity or another AI surface, and check for missing or conflicting brand claims.

This tool does not independently fact-check an AI model. It compares the answer with rules you define, so treat the output as an audit trail for human review.

Build the brand truth set

Paste the AI answer

How this AI accuracy audit tool works

AI Said What? is a free AI accuracy audit tool for checking how generative AI systems describe a brand. Instead of only tracking whether a brand appears, it compares the answer with a small truth set you define.

What is an AI brand accuracy audit?

An AI brand accuracy audit records what an AI system says about a business and compares those claims with known brand facts. Useful truth-set items include products, services, locations, pricing rules, supported markets, eligibility and other facts customers could act on.

Accuracy is different from AI visibility

A brand can be visible in AI-generated answers and still be described inaccurately. Use Signal Sample for sampled presence and citation tracking; use AI Said What? when the problem is whether the answer reflects the important facts.

How the browser-based check works

Each truth rule contains expected terms and an optional known-wrong phrase. The tool marks the rule present when one expected term appears, missing when none appear, and conflicting when the known-wrong phrase is found.

Repeat the same questions over time

Keep a stable truth set, record the prompt, surface and date, and rerun important customer questions so accuracy issues can be reproduced and discussed with evidence.

Frequently asked questions about AI accuracy audits

What is an AI accuracy audit?

It compares what AI systems say about a brand with a defined set of known facts and records missing or conflicting claims.

Does this automatically query ChatGPT or Gemini?

No. Paste the AI answer into the tool.

Can it prove an answer is true or false?

No. It checks against the rules you provide and still needs human review.

What should I put in the truth set?

Stable facts that matter if AI gets them wrong: products, locations, pricing rules, services, eligibility and markets.

Does pasted content leave my browser?

No backend is required for the audit.