Blog · measurement
How to measure AI visibility without fooling yourself
Ask an AI engine the same question two days running and roughly four in five of its sources change. That single fact invalidates most of what is sold as AI visibility reporting: the screenshot, the one-off audit, the week-on-week movement chart.
The problem
The noise floor is enormous
Before designing a measurement you have to know how much of what you would observe is noise. Two independent 2026 studies, different teams, different methods, different months, agree closely.
Asked the same question twice, ChatGPT repeated only 21.2% of its cited domains and Google AI Overviews 31.5%. Within a one-week window overlap rose only to 26.7% and 36.8%.
693,509 answers across 16,143 ChatGPT prompts and 15,805 Google prompts, 26 March to 25 April 2026. The practical consequence: a single screenshot of an AI answer is a noisy sample, not a measurement.
Day-over-day source churn ran 88.3% on Gemini, 79.2% on ChatGPT, 75.9% on Google AI Mode and 44.4% on Perplexity. Sources cited on all seven consecutive days: 11.1% on Perplexity, 2.6% on Google AI Mode, 1.1% on ChatGPT, 0.4% on Gemini. 84% of the sources for a question were cited by only one of the four engines.
530,875 citations across 181,225 distinct URLs, 2,398 real queries, seven consecutive days in June 2026, 46,259 day-over-day comparisons. Independent of the Parse study and agreeing with it closely on ChatGPT.
Three numbers to hold onto: roughly 79% of ChatGPT's sources change overnight, about 1% survive a week, and 84% of a question's sources appear on only one engine.
A screenshot is not evidence. It is one draw from a fat-tailed distribution.
The consequence
What that rules out immediately
- The one-off audit. "We asked ChatGPT ten questions and you appeared twice." At this churn that result is compatible with almost any underlying visibility rate. It is a demonstration, not a baseline.
- Position language. "You rank third for this prompt." With 1% seven-day persistence, position is not a property the system has.
- Week-on-week movement charts from small prompt sets. Movement appears every week regardless of whether anything changed. A chart that always moves cannot detect change.
- A single blended AI visibility score. With 84% of sources unique to one engine, averaging destroys the only actionable signal in the data.
- Attributing a week's change to an action. Causal claims need a control, and almost nobody running these programmes has one.
The design
What a defensible measurement looks like
The fix is not a better tool. It is treating this as sampling, which is a solved problem.
-
01
Fix the question set and freeze the wording
A rephrased question is a new series, not a continuation. If question text can be edited later, every trend line is unfalsifiable.
-
02
Write questions as buyers ask them
"Who can run AI visibility across Japanese and Korean?" is a question. "multilingual AEO services" is a keyword, and nobody types it into an assistant.
-
03
Separate retrieval from memory, and never average them
With search on, the engine cites URLs and responds to publishing in days to weeks. With search off, it answers from training data and changes only when a new model ships. Combining them makes a quarter of correct work look like failure.
-
04
Run repeatedly and report a rate, not a position
The unit is "named in 34% of runs across 40 questions over four weeks". Rates are estimable under noise; positions are not.
-
05
Measure each engine and each market separately
Engine by market is a matrix. One number is not a summary of it, it is a deletion of it.
-
06
Include control questions you do not intend to win
A question in an adjacent category you do not serve. Appearing for it is mispositioning, not progress, and it is the only cheap way to tell a real movement from the models changing underneath the benchmark.
-
07
Record mentions separately from links
Being described accurately without a link still shapes the buyer's view, and in memory mode it is the only outcome available.
-
08
Log the raw answers, not just the score
When a number moves, the only way to explain it is to read what actually changed in the text.
Sample size
How many runs is enough
There is no universal number, but the shape of the answer is straightforward. You are estimating a proportion under high variance, so precision improves with the square root of the number of observations, and observations means question times run times engine.
- 10 questions checked once gives ten observations per engine. At the observed churn that cannot distinguish 20% visibility from 40%.
- 40 questions run twice weekly for four weeks gives 320 observations per engine. Enough to see a genuine step-change and to ignore ordinary week-to-week noise.
- Perplexity needs fewer runs than Gemini. At 44.4% daily churn against 88.3%, the same confidence costs far less sampling.
The honest version of a first report. "Across 40 buyer questions run eight times on four engines, we were named in 6% of ChatGPT runs, 0% of Gemini runs, 11% of Perplexity runs and 2% of Google AI Mode runs. Confidence is lowest on Gemini because its churn is highest." That is worth acting on. "You are invisible in AI search" is not.
Why averaging is the expensive mistake
With 84% of a question's sources cited by only one engine, a blended score is an average over four systems that barely agree about anything.
The blend always looks calmer than the components, which is exactly why it gets reported. It is also why it cannot tell you what to do next.
The benchmark
What good looks like in your category
Absolute visibility numbers mean little without knowing whether anyone owns the category at all.
Across 1,094 US categories tracked in ChatGPT, 15.2% had a clear owner, 31.2% an emerging leader and 53.7% were unsettled. Clear owners kept the top spot in 90.4% of month-over-month comparisons.
50,000+ brands, 220,000+ domains, 600,000+ citations, January to June 2026. Only 11.3% of the most popular half of topics had a clear owner, against 19% in the less popular half, and that popular half carried 98% of AI search volume.
Two planning implications. In an unsettled category, and most are, a modest consistent visibility rate is a leading position, so the benchmark is the best-performing competitor rather than an absolute target. In a category with an established owner, a 90.4% month-over-month hold rate means displacement is slow and expensive, and the sensible goal is second-source presence.
Red flags
Telling a real measurement from a demo
| Claim | Why it fails | Ask instead |
|---|---|---|
| "You rank #3 for this prompt" | Answers have no stable positions; 1% of sources survive a week | "What is the rate across repeated runs?" |
| "Your AI visibility score is 42" | Blends engines that share 16% of sources | "Show me the score per engine, per market" |
| "We saw a 12% lift this week" | Within the noise floor of a small prompt set | "How many observations, and what confidence?" |
| "We can guarantee citations" | No engine offers submission or placement | "What are you contractually promising?" |
| "You appear in 0% of AI answers" | Usually a single run, often memory mode | "Which mode, how many runs, which questions?" |
| "We optimised and citations rose" | No control questions, no counterfactual | "What did the control set do?" |
Limits
What measurement cannot deliver
- No measurement makes the system deterministic. You can estimate a rate precisely. You cannot make a specific answer reproducible.
- Memory-mode absence is not fixable on a reporting cycle. It changes when a model is retrained, on the provider's schedule.
- Attribution to revenue stays hard. AI referrals are a small fraction of traffic and often arrive un-attributed, so the honest case leans on influence over the buyer's reading.
- Benchmarks age fast. Every figure on this page carries a date for that reason.
Sources
Where these numbers come from
Every figure on this page is external and linked. The studies measure different populations over different windows, so read them as converging evidence rather than one authoritative number.
- ParseAsked the same question twice, ChatGPT repeated only 21.2% of its cited domains and Google AI Overviews 31.5%. Within a one-week window overlap rose only to 26.7% and 36.8%. parse.gl ↗
- GetMentionsDay-over-day source churn ran 88.3% on Gemini, 79.2% on ChatGPT, 75.9% on Google AI Mode and 44.4% on Perplexity. Sources cited on all seven consecutive days: 11.1% on Perplexity, 2.6% on Google AI Mode, 1.1% on ChatGPT, 0.4% on Gemini. 84% of the sources for a question were cited by only one of the four engines. www.getmentions.ai ↗
- Semrush with Kevin Indig, Growth MemoAcross 1,094 US categories tracked in ChatGPT, 15.2% had a clear owner, 31.2% an emerging leader and 53.7% were unsettled. Clear owners kept the top spot in 90.4% of month-over-month comparisons. www.semrush.com ↗
- Semrush AI Visibility Index 2026ChatGPT cites an average of about 15 sources per response; Gemini cites about 3. www.semrush.com ↗
- SimilarwebChatGPT's share of generative AI web traffic fell from about 76% a year earlier to around 53%, as Gemini passed a quarter of traffic and Claude grew fastest in the category. www.similarweb.com ↗
Keep reading
FAQ
Common questions
How do you measure AI visibility?
Fix a set of buyer questions, freeze the wording, run them repeatedly against each engine in both retrieval and memory modes, and report the share of runs in which the brand is named or cited. The output is a rate with a confidence range per engine and per market, not a position or a single blended score.
Why do AI visibility tools disagree with each other?
Because they sample different questions at different times from a system with roughly 79% day-to-day source churn, and 84% of a question's sources appear on only one engine. Two honest tools running different prompt sets will legitimately produce different numbers.
How many prompts should I track?
Enough that the churn averages out. Ten prompts checked once is a demonstration. Roughly 40 buyer questions run twice weekly for a month gives a few hundred observations per engine, which is enough to separate a real step-change from ordinary noise.
Should I track retrieval mode and memory mode separately?
Always. Retrieval mode responds to what you publish within days to weeks and can cite a URL. Memory mode answers from training data, cites nothing, and only changes when a new model ships. Averaging them makes months of correct retrieval work look like failure.
Is a single AI visibility score useful?
Not on its own. Since 84% of the sources for a question are cited by only one engine, a blended score averages away the differences that determine what you would actually do next. Keep the per-engine, per-market breakdown as the reported unit.
What are control questions and why include them?
Control questions are queries in an adjacent category you deliberately do not intend to win. If you begin appearing for them, that indicates mispositioning rather than progress. They are also the cheapest way to tell a genuine movement apart from the underlying models changing beneath your benchmark.
How long before AI visibility work shows a measurable change?
On retrieval surfaces, published changes can register in days to weeks, but proving the change against this noise floor takes repeated runs across several weeks. On memory, nothing you publish moves the number until a new model is trained.
Want a baseline you could defend to a CFO? <span class='tone'>Lifewood Data Technology will build it.</span>
A frozen question set, run repeatedly across engines and markets, reported as a rate with its sample size attached.
- 100+ languages
- 50+ delivery centres
- Measured monthly
Lifewood Data Technology publishes this reference.