AI search and answers
GEO tracking: what AI visibility tools can’t measure
Written by the Top SEO Experts Team
Reviewed by Akshay VRLast updated 20 min read
Quick answer
GEO tracking tools can show whether your brand appeared in the AI answers they sampled. They can’t reliably show your position in an answer, how many people saw it, or whether it produced a lead. A January 2026 SparkToro study and Bing’s own documentation show why, so treat them as sampling instruments, not scoreboards.
Key takeaway A GEO tracker shows a sample of AI answers, not the whole picture. Read its numbers as a range, not as a rank.
A dashboard says a hypothetical plumbing company was recommended in 34% of tracked prompts in September and 21% in October. Nothing changed on the site. The owner wants to know what broke, and the honest answer may be that nothing did.
“GEO tracking” is a muddy search phrase because it also pulls in device and GPS tracking pages. Here, GEO means generative engine optimization, and tracking means AI visibility monitoring for brands that rely on search for leads. This is for owners and marketing leads deciding what to believe in a vendor demo.
We read primary sources instead of vendor claims. They are:
- The original GEO paper
- A large repeated-prompt study
- Search engine documentation
- A citation accuracy test
Some have commercial ties, and we flag each one. Our own data isn’t part of this post.
What does a GEO tracking tool actually measure?
It runs a fixed list of prompts through AI engines, saves the answers and citations, and counts how often your brand or URL appears. Everything else is derived from that sampling. This is our description of the common design, and vendors differ in the details.
Four steps sit behind most dashboards, and each hides an assumption the screen never shows:
- A prompt list, written by you or generated by the tool. If the prompts don’t match how your customers ask, the score measures something nobody searches.
- A collection method, either an engine’s API or an automated session in the consumer app. These may not return what customers see.
- A parser that decides whether a brand name or link counts as a mention or citation. A misspelled brand or a redirected URL can silently count as zero.
- A rollup into a score, share, or “position.” That’s where a sample starts to look like a ranking.
- Prompt listWritten by you or generated by the tool.
- Collection methodAn engine's API or an automated session in the consumer app.
- ParserDecides whether a brand name or link counts as a mention or citation.
- RollupTurns the sample into a score, share, or "position."
We ask about all four before we trust a chart. Sample size does the most damage, so it comes first.
Why can’t one run of a prompt tell you anything?
Because AI answers vary from run to run. Rand Fishkin of SparkToro did a study with Patrick O’Donnell of Gumshoe.ai, as Search Engine Journal reported on 30 January 2026.
What did the study find?
- ChatGPT and Google’s AI returned the same brand list less than 1% of the time.
- The same list in the same order came back in under 0.1% of runs.
- One result was useful. Visibility percentage across many runs was more consistent than ranking position, according to the researchers, who gave no single figure for it.
How was the study run?
- The study used 2,961 prompts across ChatGPT, Claude and Google’s AI Overviews, falling back to AI Mode when no Overview appeared.
- Volunteers ran them over November and December. The article gives no year, but it was published on 30 January 2026.
- Twelve recommendation prompts were each run 60 to 100 times per platform.
What are the caveats?
Watch out The work was done with Gumshoe.ai, an AI tracking vendor, and Fishkin disclosed that. It isn’t peer reviewed.
Volunteers used their own settings, which the authors chose on purpose. The study also leaves open how many runs a reliable reading needs. It leaves open whether API calls vary the way manual prompts do.
Key takeaway One screenshot, or one run per prompt, is an anecdote. A reading needs many runs, reported as a rate with a range.
How wide is the range? A worked example
This example is hypothetical, and the arithmetic is ours. Suppose a tool runs one prompt 20 times and your brand appears in 6 answers. That’s 30%.
- With 20 runs: the standard error is the square root of (0.30 × 0.70 ÷ 20), about 0.10. A rough 95% interval is 0.30 plus or minus 1.96 × 0.10, so about 10% to 50%.
- With 100 runs: run the same prompt 100 times and see 30 mentions. The standard error becomes the square root of (0.30 × 0.70 ÷ 100), about 0.046, and the interval tightens to roughly 21% to 39%.
Watch out This uses a simple normal approximation, which is rough at small samples. It also assumes independent runs, which may not hold for AI systems. Treat the interval as a floor on the uncertainty.
Definition A range, or margin, shows how far a measured share could sit above or below the number on the screen. A wider range means a less certain number.
The table sets the two hypothetical runs side by side.
| Hypothetical runs | Mentions | Share | Rough range |
|---|---|---|---|
| 20 | 6 | 30% | About 10% to 50% |
| 100 | 30 | 30% | About 21% to 39% |
Working backward gives a planning number. To pin a 30% share to within 10 points either way, you need about 81 runs. The math is 1.96 squared, times 0.30 times 0.70, divided by 0.10 squared.
Within 5 points takes about 323. Ask a vendor which of those its runs resemble.
Was the plumber’s drop real?
Take the dashboard’s 13-point drop, 34% to 21%, with about 30 runs behind each month. The standard error of the difference is the square root of (0.0865² + 0.0744²), about 0.114. The margin is roughly plus or minus 22 points, so the drop sits well inside it.
Now suppose more runs sit behind each month:
- With 200 runs, the margin shrinks to about 9 points, and a 13-point drop stands out.
- With 100 runs each, the margin is about 12 points, so the drop barely clears it.
Our view Don’t act on a month-over-month change in AI share unless the gap is larger than the margin the run count allows.
If the vendor can’t tell you the run count, you can’t do the arithmetic. The number stays a curiosity.
Can you trust “position in AI answer” metrics?
No, not as a rank. The SparkToro result above is the reason: identical lists in identical order appeared in under 0.1% of runs. Fishkin called any tool offering ranking position in AI answers “full of baloney,” and told readers to make sure a provider “shows their math.”
What do the engines themselves say?
Engines are cautious too. Bing introduced its AI Performance report in public preview on the Bing Webmaster Blog. The report says its Total Citations figure is shown “without indicating placement or presentation within a specific answer.”
Average Cited Pages “does not indicate ranking, authority, or the role of any page within an individual answer.”
A first-party report, with access to the real answers, declines to give you a position. A third-party tool working from samples that offers one owes you an explanation.
Our view We ignore any “average position” number from an AI visibility tool. What we report is the share of prompts in which you appear, with a run count and a range.
Do tools see what real customers see?
Not necessarily. Many tools collect answers through an engine’s API, while customers use the app. One vendor study suggests those can behave differently, but it’s a single, unreplicated result.
What did Genezio find?
Genezio ran identical UK banking scenarios through the GPT-5.2 API and the ChatGPT.com interface. The run covered the 27 days before 4 February 2026, and Genezio published the comparison. Its finding: zero overlap in the search-style queries each path generated.
- Of 3,856 unique queries, 2,785 (72.2%) appeared only in the interface and 1,071 (27.8%) only in the API.
- Of 5,701 unique source URLs, 704 (12.3%) were cited by both.
Watch out Genezio notes it’s a single sector, uses a “lightweight” domain classification, and reports no significance testing. It also sells an AI visibility platform and recommends scoring mainly from interface runs.
How can you check your own tool?
So read the finding as a reason to ask, not as proof. If a tool collects through an API, ask whether it has validated that against the consumer app for prompts like yours.
You can run a crude version of that check yourself:
- Take 10 of your prompts.
- Run each once in a logged-out session of the consumer app.
- Compare the brands and links with what the tool recorded that day.
Expect differences. If the two almost never agree, the tool describes a different product than your customers use.
We haven’t found published data on how location, account history or memory features change answers for local service queries. That’s an unknown, and a real one for a business that serves one metro.
Want to apply this to your site?
Not sure where to start? Get a plan for your own business.
Do the GEO research papers prove the tactics work?
They show something narrower. The paper that named the field is “GEO: Generative Engine Optimization” by Pranjal Aggarwal and five co-authors. It was first posted on arXiv on 16 November 2023, and the current version is dated 28 June 2024.
It claims GEO methods can “boost visibility by up to 40%.”
We read the abstract, the benchmark design and the main results table. We didn’t review the later sections, so this is a reading of the method and headline results only.
What did the paper do?
What the paper did, as far as we read it:
- Built GEO-bench, 10,000 queries (8,000 train, 1,000 validation, 1,000 test) from nine datasets, mostly informational.
- Tested methods on the authors’ own engine, which retrieves the top five Google results and generates an answer with gpt-3.5-turbo at temperature 0.7. That approximates commercial engines. It isn’t one of them.
- Measured visibility with position-adjusted word count, plus an LLM-scored “subjective impression” that the authors say is poorly calibrated and had to be rescaled.
- Reported that quotation addition, statistics addition and fluency optimization scored above baseline. Keyword stuffing scored below baseline on position-adjusted word count, and the paper calls it and unique words non-performing.
The paper also tested on Perplexity.ai “in the wild” and reports gains of up to 37%. The abstract says effectiveness varies by domain.
Our view The paper is a good source of hypotheses. It isn’t evidence that a tactic lifts citations for a plumber in Phoenix today.
Why is the evidence limited?
- The test engine is a simplified stand-in running an older model.
- The scoring leans on an LLM judge.
- Each query optimizes one source in isolation, which isn’t how competitors behave.
We label the paper’s tactics experimental. Test them on your own pages before rolling them into a template. Our post on LLM SEO myths covers the claims we see repeated most.
Can a tool tell you whether AI answers about you are correct?
Mostly no. Counting mentions isn’t the same as checking them.
What did the Tow Center test?
The Tow Center for Digital Journalism at Columbia tested citation accuracy in March 2025. It used eight AI search engines and 200 queries for each, 1,600 in total. The Tow Center published the report on 6 March 2025, and Nieman Lab covered it.
Each query gave an engine a quote from a news article and asked for the title, date, publication and URL.
What did it find?
The report says the engines “provided incorrect answers to more than 60 percent of queries.” ChatGPT Search incorrectly identified 134 of its 200 articles and signaled uncertainty just fifteen times.
The scope is narrow. Every test quote came from a news article, so the result speaks to news citation, not to a plumber’s service page. It still shows that an engine can cite confidently and be wrong, and a presence counter would score that as a win.
How do you check accuracy yourself?
If a tool reports sentiment or accuracy, ask how. Our practical check is manual and light.
Each month, read a sample of the actual answers about your business and mark each fact right, wrong or missing:
- Services you offer and services you don’t.
- Service area and the cities you actually cover.
- Hours, phone number and address.
- Any price, guarantee or license claim attributed to you.
Wrong answers matter most where a wrong answer sends a customer elsewhere, such as a listed service you stopped offering. Keep the dated log, because it’s the only accuracy record you’ll have.
Quick check This month, read a sample of the answers about your business. Mark each fact right, wrong or missing, and date the log.
What can you measure reliably?
Clicks, crawls and conversations. These signals come from systems with first-party access, and each is documented. None of them shows how often you were mentioned without a click.
| Signal | Where it comes from | What it shows | What it doesn’t |
|---|---|---|---|
| Google Search Console, Web search type | Google’s AI features documentation says cited sites count toward overall Web traffic | Clicks and impressions by query and page | A separate AI-feature view; Google describes none |
| Bing Webmaster Tools, AI Performance (public preview) | Bing says it covers Microsoft Copilot, AI-generated summaries in Bing and select partner integrations | Total Citations, Average Cited Pages, grounding queries (a sample), page-level citations | Placement, ranking, authority or role in an answer |
| Analytics referrals | OpenAI’s publishers FAQ: ChatGPT adds utm_source=chatgpt.com | Visits that clicked through from ChatGPT search | Mentions with no click, or a later branded search |
| Crawler access | robots.txt and server logs | Whether crawlers such as OAI-SearchBot can fetch your pages | Whether the page is cited |
| Sales intake | A “how did you hear about us” field | What customers say about AI assistants | Anything customers forget or misattribute |
What do the primary pages add?
Two details from the primary pages are worth knowing. Bing calls its report “an early step toward Generative Engine Optimization (GEO) tooling,” says the grounding query data “represents a sample of overall citation activity,” and aggregates across supported AI surfaces.
OpenAI’s page says to make sure you aren’t blocking OAI-SearchBot if you want to appear in ChatGPT search. It also notes that a noindex meta tag only works if the crawler is allowed to crawl the page.
Our guide to measuring AI visibility and post on tracking AI referral traffic in GA4 go through the setup. For Microsoft’s side, see our Copilot visibility post.
Key takeaway The weakest link in that table is the last row, and it’s the one closest to revenue. Add the question to your call script and your form before you buy any software.
How should a service business use a GEO tracker?
As a sampling instrument, with its method disclosed, alongside outcome data. This is the order of operations we’d follow, and it’s our view, not an industry standard.
- Fix crawl access and indexing first. A tool can’t measure visibility for pages engines can’t read. Our free Website SEO Checker covers the basics.
- Write 30 to 50 prompts from real customer questions: sales calls, form submissions, Google Business Profile questions. Freeze the list for a quarter.
- Sort prompts into buckets such as problem questions, comparison questions, “near me” questions and branded questions. Report each bucket separately.
- Report share of prompts with a mention, with the run count and a range. Drop position.
- Pair the numbers with calls, form fills and the “how did you hear” answers, month by month.
- Change one thing at a time. If you rewrite pages and swap prompts together, you learn nothing.
- Fix crawl access and indexingA tool can't measure pages engines can't read.
- Write the prompt listUse real customer questions and freeze the list for a quarter.
- Sort prompts into bucketsReport each bucket separately.
- Report share with a rangeInclude the run count. Drop position.
- Pair with outcomesCompare to calls, form fills and "how did you hear" answers.
- Change one thing at a timeOtherwise you learn nothing.
How do we read a monthly report?
When a monthly AI visibility report lands on the desk, we apply three rules:
- If the change in share is inside the margin, write “no change” and move on.
- If share moves outside the margin and calls move the same way, look at what changed on the site that month.
- If share moves and calls don’t, hold your spending. Mentions without leads are a hypothesis, not a result.
What should you ask a vendor?
Before paying any vendor, put these questions to them in writing and expect specific answers:
- How many runs per prompt, per engine, per period?
- API or consumer app, and has that been validated against what customers see?
- Which location, language and logged-in state do the runs use?
- How does the parser decide a mention or citation counts?
- Can I export raw answers with dates so I can check them?
- What happens to my score when you change the prompt set or the engine version?
If a vendor can’t answer the third and fifth, treat the dashboard as a story.
For how assistants choose what to cite, see how AI assistants choose sources. For Google’s side, see our post on AI Overviews on legal and health queries.
To improve visibility rather than just count it, start with our AI search optimization service. The guide to getting cited by AI search is another place to begin.
What does the evidence not show?
Several things stay open. As of 22 October 2026, we couldn’t find these in the sources we opened.
- How many runs a reliable AI visibility reading needs. The SparkToro study, as reported, lists it as open.
- Whether API-collected answers match what customers see, beyond one vendor’s single-sector study.
- Whether any tactic from the GEO paper lifts citations on live engines for local service queries.
- How personalization, location or memory change answers for “near me” and service-area queries.
- Whether an AI mention, with no click, produces a lead. No public data we found ties the two together.
- Whether the Tow Center’s news citation failure rates carry over to business or service queries.
If you need the term defined, our AI visibility glossary entry has it.
Frequently asked questions
What is GEO tracking?
Here GEO means generative engine optimization, not device location. GEO tracking is software that sends a fixed list of prompts to AI engines such as ChatGPT or Google’s AI Mode.
It stores the answers and citations, and counts how often a brand or URL appears. Vendors convert those counts into visibility scores.
Are AI visibility scores accurate?
Only as a sample of the prompts and engines the tool tested. SparkToro and Gumshoe.ai found ChatGPT and Google’s AI returned the same brand list in under 1% of repeated runs.
So a score from a handful of runs can swing widely. Ask for the run count and a range first.
Can I track my rank inside ChatGPT or AI Overviews?
Not in a way we’d trust. The same SparkToro study found identical lists in identical order in under 0.1% of runs.
Bing’s AI Performance report also says its citation counts don’t indicate placement. Track how often you appear across many runs rather than a position number.
How do I see traffic from ChatGPT in analytics?
OpenAI’s publishers FAQ says ChatGPT adds utm_source=chatgpt.com to referral URLs, so a tool like Google Analytics can separate those visits. That covers clicks only. A person who read an answer, remembered your name and searched for you later will land in branded or direct traffic.
Is AI Mode traffic reported separately in Search Console?
Google’s documentation says sites cited in AI features count toward overall Web search traffic in the Performance report. It describes no separate view for them.
We haven’t found a dependable way to isolate AI clicks there, so compare query and page trends before and after any change you make.
Should a small service business pay for a GEO tool?
Usually not first. Fix crawl access, indexing and the pages that win calls, then keep a small manual prompt log.
Consider a tool only if the vendor discloses runs per prompt, engine access, location and detection method. Also be sure that someone on your team will act on the data every month.
Next step
Check the crawl and indexing basics before you measure anything else. Start with the free Website SEO Checker, then Get Your Organic Growth Plan.
Sources
- Search Engine Journal, AI recommendations change with nearly every query: SparkToro, accessed 22 Oct 2026.
- Aggarwal et al., GEO: Generative Engine Optimization, arXiv:2311.09735v3, accessed 22 Oct 2026.
- Microsoft Bing Webmaster Blog, Introducing AI Performance in Bing Webmaster Tools Public Preview, accessed 22 Oct 2026.
- Genezio, Zero query overlaps between GPT-5.2 (API) and ChatGPT.com, accessed 22 Oct 2026.
- Columbia Journalism Review (Tow Center), AI Search Has a Citation Problem, 6 March 2025, accessed 23 October 2026.
- Nieman Lab, AI search engines fail to produce accurate citations in over 60% of tests, according to new Tow Center study, accessed 22 Oct 2026.
- OpenAI Help Center, Publishers and developers FAQ, accessed 22 Oct 2026.
- Google Search Central, AI features and your website, last updated 10 Dec 2025 on the page, accessed 22 Oct 2026.
Want to apply this to your site?
Not sure where to start? Get a plan for your own business.
