AI search and answers
AI search for SaaS: how to test comparison prompts
Written by the Top SEO Experts Team
Reviewed by Siddharth GangalLast updated 21 min read
Quick answer
Run fixed comparison prompts (“X vs Y,” “best X for Y,” “alternatives to Z”) many times across several engines. Log whether your SaaS is named, how it is described and which sources are cited. Report rates with a range, not ranks, because repeated runs rarely match.
Here’s the trap, in three short steps:
- A founder asks ChatGPT for the best tools in their category.
- A rival is named first, so the founder spends the quarter reacting.
- An hour later, the same question gives a different list.
One screenshot proves almost nothing.
Quick check Can you say how many runs sit behind the last AI answer you saw about your product? If the answer is one, treat it as a story, not a result.
What does “AI search SaaS” mean here?
The phrase “AI search SaaS” is muddy. It also covers SaaS products that add AI search to their own apps. This post is about the other meaning: AI systems answering buyers’ questions about SaaS products.
It’s for founders and marketing leads. They want to know if ChatGPT, Google’s AI features or Copilot recommend their product.
What won’t you find here?
We haven’t run a published test of our own, so there are no first-party results here. You get the method we’d use. You also get the third-party evidence we could verify, and a plain list of what nobody has proven.
What are comparison prompts, and why do they matter for SaaS?
A comparison prompt is a question a buyer asks to narrow a shortlist. An example is “Is Tool A better than Tool B for a 10-person team?”
These prompts come late in the buying process. The answer may name your product, name a rival, or name neither.
Definition A comparison prompt asks an AI engine to compare, rank or suggest software, such as a head-to-head question or a best-for question.
We sort them into three types. Each one tests something different about how an engine treats you.
| Prompt type | Example | What it tests |
|---|---|---|
| Head-to-head | “Tool A vs Tool B” | Whether the engine describes you correctly |
| Best-for | “best CRM for a five-person sales team” | Whether you’re in the consideration set at all |
| Alternatives | “cheaper options than Tool A” | Who the engine thinks you replace |
The grouping is our own framework, not an industry standard. We also have no data on what share of SaaS buyers shortlist with AI engines.
So don’t use the prompts to forecast leads. Use them to see how you’re described.
Why can’t you judge your visibility from one prompt?
Because the same prompt rarely returns the same answer twice. Rand Fishkin of SparkToro and Patrick O’Donnell of Gumshoe.ai ran a study that Search Engine Journal reported on 30 January 2026. ChatGPT and Google’s AI each returned the same brand list in under 1% of repeated runs.
What did the SparkToro study find?
The same list in the same order showed up less than 0.1% of the time. The study ran 12 brand-recommendation prompts 60 to 100 times per platform across ChatGPT, Claude and Google. Volunteers ran them over November and December, according to the article (published 30 January 2026).
What held up better was how often a brand appeared across many runs. In the headphone prompts, a handful of big brands appeared in roughly 55% to 77% of responses. The researchers concluded that a visibility percentage is steadier than a rank position.
What are the limits of that study?
The article states its own limits. Gumshoe.ai sells AI tracking tools, and the study isn’t peer reviewed. The authors also list open questions, including how many runs a reliable reading needs.
So a single answer, good or bad, is an anecdote. If someone shows you one, ask how many runs sit behind it.
Key takeaway One AI answer is an anecdote. How often a brand appears across many runs is a steadier measure than its rank in any single answer.
Who do AI engines actually cite when recommending software?
In the one study we could verify, the answer was mostly vendor sites and small blogs. Review platforms were nearly absent. The study is narrow, so treat it as a lead to test, not a rule.
How was the DerivateX study run?
It comes from DerivateX, an SEO and GEO agency. Its report is dated 18 June 2026.
The method was simple. It used 40 B2B SaaS categories and one buyer-intent prompt per category. ChatGPT had web search on, and each prompt ran in a fresh temporary chat.
What did it find?
The study logged 233 recommendations. Of these, 188 carried a clickable third-party citation across 105 unique domains. Here is the split by source type.
| Source type | Share of citations | Citations |
|---|---|---|
| Vendor-owned content | 51.1% | 96 |
| Independent and niche sites | 22.9% | 43 |
| Recognized media | 14.9% | 28 |
| Community sites such as Reddit and LinkedIn | 9.6% | 18 |
| Review aggregators | 1.1% | 2 |
| Unclassified | 0.5% | One |
G2 and Capterra received none.
What should you keep in mind before using it?
Before you lean on that, note the limits:
- It covers one engine and one query type, which is vendor discovery.
- It used one prompt per category, so there’s no run-to-run variation to speak of.
- Domains were classified by hand, with no reported check between reviewers.
DerivateX also sells services to B2B SaaS companies. Its report frames vendor-owned content as the opportunity. That doesn’t make the numbers wrong, but read the framing separately from the counts.
What you can take from it is a hypothesis for your category. In that sample, vendor pages were cited far more often than review sites. It says nothing about what any engine will do with your product.
Watch out One agency study, one engine and one prompt per category cannot show what any engine will do with your product. Use it as a lead to test, not a rule.
What does Google say about appearing in AI features?
Google says there are no extra requirements. A page must be indexed and eligible to show in Search with a snippet.
The page is AI features and your website. It was last updated 10 December 2025 when we read it. It says there are no additional requirements “nor other special optimizations necessary.”
What does Google say you don’t need?
The page is more specific than that. It says there’s no special schema.org markup to add. You also don’t need new machine-readable files or AI text files.
What are the listed best practices?
The listed best practices are mundane:
- Allow crawling in robots.txt and in any CDN or hosting layer.
- Link to content internally.
- Keep important content in text.
- Make structured data match the visible page.
Which details matter most for SaaS sites?
Two details matter for SaaS sites. First is the CDN line.
In our reading, bot protection and firewall rules can block crawlers at the infrastructure level while robots.txt looks fine. Second, if you use nosnippet or max-snippet controls on marketing pages, Google says those limit what’s shown from your pages in Search.
Definition A CDN is a network that serves your site’s files from servers near the visitor.
The page also says AI Overviews and AI Mode may use “query fan-out,” which issues several related searches across subtopics. This can surface a wider set of supporting pages. It also states plainly that indexing and serving isn’t guaranteed, even for pages that meet every requirement.
For measurement, Google says sites in AI features are counted in the Performance report under the Web search type. We found no separate AI report described there.
What do OpenAI and Microsoft say?
Two other engines publish details that matter to a SaaS site.
OpenAI’s publishers FAQ says to make sure you aren’t blocking OAI-SearchBot if you want content in ChatGPT summaries and snippets. The same FAQ says ChatGPT adds utm_source=chatgpt.com to referral URLs. It treats GPTBot as the separate user-agent to disallow for training.
Microsoft’s AI Performance report in Bing Webmaster Tools was introduced in public preview in February 2026. It reports total citations, average cited pages and sampled grounding queries for Copilot, Bing AI summaries and select partners. Microsoft says the numbers don’t indicate placement.
Our view Check robots.txt and your firewall for accidental blocks before you test anything. A test on a site the engine can’t reach measures nothing.
Watch out Google’s noindex documentation says the page must not be blocked by a robots.txt file and must be otherwise accessible to the crawler. A crawler that can’t reach a page can’t read the noindex tag on it.
How do you run a comparison prompt test?
Build a fixed prompt set. Run every prompt many times per engine under logged conditions. Record mentions, descriptions and citations in a table you can re-run later.
Our method has six steps. It’s ours, not a documented standard.
- Write the promptsUse your buyers' words and all three prompt types.
- Fix the settingsChoose engines and record how each one is set up.
- Run each prompt many timesUse a fresh chat for every run.
- Log the same fieldsSave full answers, cited URLs and a correctness rubric.
- Score rates, not ranksWork out mention, accuracy and citation rates per engine.
- Repeat and compareRe-run monthly and after big changes.
Step 1: write 15 to 30 prompts
Use the three types above, in your buyers’ words. Pull wording from sales calls, support tickets and Search Console queries.
Include prompts with no brand named. They show whether you’re in the set at all.
Here’s a slice of a hypothetical prompt set for an invoicing tool we’ll call Tool A:
| ID | Type | Prompt |
|---|---|---|
| P01 | Best-for | best invoicing software for freelancers |
| P02 | Best-for | invoicing tool that connects to QuickBooks for a 5-person agency |
| P03 | Head-to-head | Tool A vs Tool B for recurring invoices |
| P04 | Head-to-head | is Tool A worth it compared to Tool C |
| P05 | Alternatives | alternatives to Tool B with lower pricing |
| P06 | Alternatives | Tool A alternatives for teams that bill by the hour |
Notice P02. An integration and a team size are how real buyers narrow a list. They give you a sharper read than the generic “best invoicing software.”
Step 2: choose engines and fix the settings
Pick the two to four your buyers plausibly use. Examples are ChatGPT with search, Google’s AI Overviews or AI Mode, and Copilot.
For each engine, record these four things:
- Logged-in or logged-out
- Location
- Whether web search was on
- The model name shown
Step 3: run each prompt at least 10 times per engine
More is better for decisions. Use a fresh chat for each run so earlier answers don’t leak in.
Save the full answer text and the cited URLs, not a summary.
Step 4: log the same fields every time
Here’s a header you can paste into a sheet:
date,engine,settings,prompt_id,prompt_type,run_no,brand_named,described_correctly,competitors_named,cited_urls,notesFor described_correctly, decide the rubric before the first run. Ours checks four facts against your own site. They are pricing model, plan limits, integrations and who the product is for.
One wrong fact among the four makes the run “partly correct.” Two or more makes it “wrong.”
Step 5: score rates, not ranks
For each prompt and engine, calculate three rates. Never average across engines.
- Mention rateRuns where you're named divided by total runs.
- Accuracy rateNamed runs where the description is right.
- Citation rateRuns where your own domain is cited.
Step 6: repeat on a schedule and after changes
Re-run the whole set monthly, and after any major change to your site or your third-party coverage. Keep raw answers with dates so you can see what changed.
Want to apply this to your site?
Not sure where to start? Get a plan for your own business.
How many runs is enough?
Enough that the range is narrower than the decision you want to make. Small samples give wide ranges. So a mention rate from 10 runs is a hint, not a result.
We suggest 10 runs per prompt and engine for a first read. Run 30 or more before you change strategy.
What does the arithmetic look like?
Here’s the arithmetic for a hypothetical SaaS product. Named in 3 of 10 runs, it scores 30%, with a Wilson 95% interval of roughly 11% to 60%. Named in 30 of 100 runs, it also scores 30%, but the range tightens to roughly 22% to 40%.
Definition A Wilson interval is a way to show the range of true rates that fit a small sample.
The same logic applies to zero. Zero mentions in 10 runs is consistent with a true rate up to about 28%. Zero in 30 runs is consistent with up to about 11%.
Did our change work?
Now take the question teams ask most: did our change work? Say a hypothetical product was named in 4 of 10 runs before a comparison-page update. That is 40%, with an interval of about 17% to 69%.
After the update it was named in 7 of 10 runs. That is 70%, with an interval of about 40% to 89%.
That looks like a 30-point jump. The ranges overlap almost completely. So treat it as noise until more runs say otherwise.
Example In this hypothetical case, a product named in 4 of 10 runs and then 7 of 10 after an update shows a 30-point jump. The ranges overlap almost completely, so treat it as noise.
Scale the same pair up. At 12 of 30 versus 21 of 30, the ranges are about 25% to 58% and 52% to 83%. They still overlap slightly.
At 40 of 100 versus 70 of 100, the ranges separate. They are about 31% to 50% against 60% to 78%.
Our view Overlap is a rough guide, not a formal test, but it’s a good brake on celebrating early.
This hypothetical table collects the examples above in one place.
| Hypothetical case | Named in | Wilson 95% range |
|---|---|---|
| Small test | 3 of 10 runs | 11% to 60% |
| Bigger test | 30 of 100 runs | 22% to 40% |
| Before update, 10 runs | 4 of 10 | 17% to 69% |
| After update, 10 runs | 7 of 10 | 40% to 89% |
| Before update, 30 runs | 12 of 30 | 25% to 58% |
| After update, 30 runs | 21 of 30 | 52% to 83% |
| Before update, 100 runs | 40 of 100 | 31% to 50% |
| After update, 100 runs | 70 of 100 | 60% to 78% |
These numbers are illustrative and the intervals are our calculation. Nobody has published a standard run count for AI visibility. The SparkToro authors list it as an open question.
How long does it take?
Here’s a practical note on time. Say you run prompts by hand, and each takes about a minute (our guess). Then 15 prompts on two engines at 10 runs each is 300 runs, or roughly five hours.
Budget for that, or automate with care. API results may differ from what buyers see in the apps.
What do you do with the results?
Match the finding to the fix, and keep the order.
Our view Fix access and facts first. Then fix comparison content, then third-party coverage, then measurement.
| What you see | Where to start |
|---|---|
| The engine can’t reach or index your pages | Fix crawl access and indexing |
| You’re named but described wrongly | Fix your own pages |
| You’re not named in best-for prompts | Look at which sources the engine cites |
| You’re named in best-for but lose in head-to-head | Publish an honest comparison page |
If the engine can’t reach or index your pages
Fix crawl access and indexing first. Google says a page must be indexed and snippet-eligible to appear in its AI features. Check robots.txt, noindex tags, CDN rules and rendering.
If you’re named but described wrongly
Old pricing, missing integrations or wrong plan limits are the usual signs. Fix your own pages. Put pricing, features and integrations in plain text on crawlable pages with a visible date.
If you’re not named in best-for prompts
Look at which sources the engine cites. If they’re third-party lists or community threads, your gap may be coverage there. If they’re competitors’ own pages, publish accurate pages of your own.
If you’re named in best-for but lose in head-to-head
Publish an honest comparison page. State where the competitor is stronger, date it, and keep it current.
How many comparison pages should you publish?
On comparison pages, resist volume. Hundreds of templated “X vs Y” pages are a bad bet. A smaller set you can keep accurate is safer.
Our guide to programmatic SEO for SaaS explains where scale helps and where it backfires.
For the content side, see our posts on SaaS content marketing and SaaS SEO KPIs. Our post on GEO tracking limits covers what visibility tools can’t tell you.
What mistakes make a prompt test useless?
Most failures come from test design, not the engine. These are the ones to watch for, drawn from the evidence above and from how sampling works.
- Testing one run. It’s an anecdote, as covered above.
- Testing only branded prompts. Asking about your own name tells you nothing about being discovered.
- Blending engines into one score. A single number hides which engine moved.
- Editing the prompt set mid-test. If you change prompts, start a new series.
- Chasing position. In the SparkToro study, order varied far more than visibility did. Bing’s own report also says citation counts don’t show placement.
- Treating a mention as a lead. We found no public data linking an AI mention, with no click, to revenue. Watch branded search and direct traffic, and ask new customers how they found you.
- Polishing the test instead of the product pages. A vague pricing page isn’t fixed by a better dashboard.
What is still unproven?
A lot. As of 22 October 2026, this is the list we’d keep in front of any client.
| Status | What it covers |
|---|---|
| Documented | Google requires indexing and snippet eligibility for AI features and describes no special optimization. OpenAI says allowing OAI-SearchBot is needed for inclusion in ChatGPT summaries and snippets. |
| Published but limited | The SparkToro and DerivateX data, with the vendor interests and sample limits described above. |
| Experimental | The GEO paper by Aggarwal and colleagues, described below. |
| Unknown | Our own results. We haven’t published a dated test, and we aren’t borrowing anyone else’s. |
What does the GEO paper say?
The GEO paper by Aggarwal and colleagues was submitted in November 2023 and revised June 2024. It reports that some content changes can raise visibility “by up to 40%” in its own test setup. It also says results vary by domain.
We treat it as a source of hypotheses, not proof about live engines.
What is unproven?
Unproven, and we’d tell any client the same:
- That any content tactic reliably raises your mention rate in ChatGPT, Gemini, Copilot or Perplexity for SaaS comparison prompts.
- That review sites, directories or Reddit threads move AI recommendations for your category, in either direction.
- That an AI mention without a click produces pipeline.
- How many runs a reliable reading needs, and whether API runs match what buyers see in the apps.
When we publish our own test, it will state the prompt set, engines, settings, run counts and date.
For definitions, see our glossary entry on AI Overviews. For the wider debate on what’s real in this field, read our post on LLM SEO myths.
What should a SaaS team do this month?
Start small and make it repeatable. Pick 15 prompts, run each 10 times on two engines, and log the results in a sheet. That’s 300 runs and a baseline you can defend.
At the same time, do the dull checks that the documents we read actually support. Do them in this order.
- Check indexing and crawlingConfirm marketing pages are indexed and crawlable, including for OAI-SearchBot and through your CDN.
- Put key facts in plain textShow pricing, plans, integrations and limits on pages with a visible update date.
- Read how engines describe rivalsList the five competitors your buyers compare you with.
- Check your analyticsLook for utm_source=chatgpt.com referrals.
- Pick your comparison pagesChoose the two or three you can keep accurate, and write those first.
If you want help turning this into a program, our AI search optimization service covers measurement and the content behind it. Our guides on AI search optimization, getting cited by AI search and ranking in AI Overviews go step by step. Our SaaS SEO page shows the wider plan.
Frequently asked questions
How do I check whether ChatGPT recommends my SaaS product?
Write 15 to 30 prompts in your buyers’ words, such as best tools for your category or your product versus a competitor. Run each at least 10 times per engine in fresh chats. Log whether you’re named, how you’re described and which sources appear, then report a mention rate with a range.
Why do I get different answers every time I ask?
AI answers are generated fresh on each run, and a SparkToro and Gumshoe.ai study reported on 30 January 2026 tested this. Repeated runs of the same prompt returned the same brand list in under 1% of cases. That’s why we measure how often a brand shows up across many runs.
Do G2 and Capterra reviews help me get recommended by AI?
We couldn’t confirm it. One agency study of 40 B2B SaaS categories saw ChatGPT with web search cite neither site, using one prompt per category and one engine. Reviews still matter to buyers directly, but their effect on AI answers is unproven.
Can I see AI search traffic in Google Search Console?
Only inside your normal Web totals. Google’s documentation says sites in AI features are counted in the Performance report under the Web search type, and it describes no separate AI report. For ChatGPT, OpenAI says it adds utm_source=chatgpt.com to referral URLs, so analytics can isolate those visits.
Should I publish comparison pages against my competitors?
Yes, if the pages are accurate, dated and fair about where a competitor is stronger. Buyers search for them, and AI engines may draw on them, but skip templated bulk pages with thin claims. A smaller set you can keep current is safer and more useful to readers.
Is there anything that guarantees my SaaS gets cited by AI search?
No. Google says there are no additional requirements or special optimizations for AI Overviews and AI Mode beyond being indexed and snippet-eligible. It also says indexing and serving isn’t guaranteed, so anyone promising citations is selling a guess. You can improve how findable and accurately described you are, and then measure it.
Before you test prompts, check that the pages behind them are reachable and indexable. The free Website SEO Checker can help. Then Get Your Organic Growth Plan.
Sources
- Search Engine Journal, AI recommendations change with nearly every query: SparkToro, accessed 22 Oct 2026.
- DerivateX, ChatGPT software recommendation sources report, accessed 22 Oct 2026.
- Google Search Central, AI features and your website, accessed 22 Oct 2026.
- Google Search Central, Block Search indexing with noindex, accessed 23 October 2026.
- OpenAI Help Center, Publishers and developers FAQ, accessed 22 Oct 2026.
- Microsoft Bing Webmaster Blog, Introducing AI Performance in Bing Webmaster Tools Public Preview, accessed 22 Oct 2026.
- Aggarwal et al., GEO: Generative Engine Optimization, arXiv:2311.09735v3, accessed 22 Oct 2026.
Want to apply this to your site?
Not sure where to start? Get a plan for your own business.
