When an agency sends you an “AI visibility” report, check four things before you trust it: which prompts were tested and who chose them, how many times each was run, whether it reports mention rate rather than a single “AI rank”, and whether it links visibility to leads or revenue. A report that cannot answer these is a screenshot, not a measurement.
Key takeaways
- AI answers change on almost every run, so one-off rankings in ChatGPT or Perplexity are mostly noise.
- Ask for the full prompt list, run counts, dates, tools and settings. If the agency will not share them, discount the report.
- A good report shows mention rate by question type, accuracy of facts about you, cited sources and actions taken.
- Use the 8-question checklist and the “good answer vs red flag” table below in your next review call.
This is the buyer’s side of our B2B GEO and AI search visibility guide. If you want to run a check yourself first, see how to test your brand in ChatGPT and Perplexity.
Why AI visibility reports are easy to get wrong
Classic SEO reports rest on rankings that stay fairly stable from one day to the next. AI answers do not. In a 2026 study by SparkToro and Gumshoe.ai, the same prompts run again and again almost never produced the same list of brands, or the same order, as Search Engine Land reported. So a slide that says “you rank #3 in ChatGPT” may simply describe one lucky run. The study’s authors suggest that how often a brand appears across many runs is a more useful measure than its position.
The 8 questions to ask about any AI visibility report
| Ask | Good answer | Red flag |
|---|---|---|
| 1. Which prompts did you test? | Full list shared, written as buyers ask | “Proprietary”, or only branded prompts |
| 2. Who chose them? | Agreed with you, from sales and customer questions | Chosen by the agency, never reviewed |
| 3. How many runs per prompt? | Several runs per prompt, per tool | One run, one screenshot |
| 4. Which tools and settings? | Named tools, dates, logged-out or memory off, location | “AI search” with no detail |
| 5. What is the headline metric? | Mention rate by question type | A single “AI rank” or “visibility score” with no formula |
| 6. Are the facts about us right? | Accuracy checked: price, services, locations | Not checked |
| 7. Which sources do the answers cite? | List of cited sites, with gaps we can act on | No source data |
| 8. What changed in leads or revenue? | AI referral visits and leads tracked, with caveats | No link to business results |
Score the report out of 16
2 points per question for a good answer, 1 for partial, 0 for a red flag. 13–16: trust it. 8–12: ask for fixes. 0–7: treat it as a sales deck.
Check whether your pages are easy for AI to quote
Whatever the report says, your own pages need clear answers and facts. Score one with our Answer Readiness Checker.
What a useful monthly AI visibility report contains
- Method box: prompt list, runs per prompt, tools, dates and settings, unchanged month to month.
- Mention rate by question type: category discovery, problem, comparison and brand questions, with last month alongside.
- Accuracy log: wrong facts found about you and where they come from.
- Source list: the sites AI answers cite for your target questions, and which ones mention you.
- Actions: what was done (pages updated, listings fixed, outreach) and what is next.
- Business view: visits and leads from AI tools in your analytics, labelled as partial, since not every AI visit passes a referrer. Our AI visibility tracking guide covers the setup.
Example (illustrative)
A Bengaluru SaaS firm’s agency report showed a “visibility score” rising from 42 to 61. On the review call, the agency could not share the prompt list and had run each prompt once. After switching to a fixed 20-prompt set with three runs, the firm’s real mention rate for category questions was 15%, and the work moved to the review sites that AI answers were citing.
When to push back, and how
Do not cancel on a single bad report. Ask for the method in writing, agree a fixed prompt set together, and give it two monthly cycles. If the method still cannot be shared, or the numbers never connect to visits or leads, it is fair to stop paying for that line item.
The Bargaon view
AI visibility is worth measuring, but only with an open method. Ask the eight questions, score the report, and insist on mention rate, accuracy and sources over a single score. For the bigger picture, see our B2B GEO guide; if you want a report built this way, our GEO service reports on exactly these points.
Start with the pages AI should be quoting
Run your key page through the Answer Readiness Checker, then see our GEO service for an open-method AI visibility report.
FAQs
Is an “AI rank” in ChatGPT a reliable number?
Not on its own. AI answers vary from run to run, so a position from one run says little. Mention rate across many runs of the same prompts is more reliable.
How many prompts should an AI visibility report cover?
Enough to cover your main question types; around 20 to start is practical. What matters more is that the same set is used every month and each prompt is run more than once.
Can AI visibility be linked to leads?
Partly. Some AI tools pass referral data you can see in analytics, but not all visits are labelled. Treat AI-referred leads as a floor, not the full picture.
Sources
- Search Engine Land: AI recommendation lists repeat less than 1% of the time (2026, on the SparkToro and Gumshoe.ai study)
Last updated: 7 November 2026. How we researched: published reporting on the SparkToro and Gumshoe.ai study, plus Bargaon’s own review checklist and scorecard described above.
Digital Marketing & SEO Specialist
Himanshu Patni has spent more than nine years in search and growth marketing, watching discovery move from ten blue links to AI answers. His work brings SEO, AEO and GEO together with paid media and measurement as one system. At Bargaon, he writes about where search is heading and what businesses should do before the shift becomes obvious.
