SEO case study red flags: how to check if one is real

Below are the questions to ask, a checklist to run on any proposal, and what a checklist can’t prove.

We write SEO case studies ourselves, so we have a stake here and we aren’t neutral. We’ve tried to hold our own pages to the same standard, and the section near the end says where they fall short.

What makes an SEO case study trustworthy?

A trustworthy case study is one a skeptical stranger could check. It states the site, the time window, the metric, the data source, the work done and the parts not disclosed. Each missing piece is a place where an inflated claim can hide.

What does Google say about case studies?

Google’s advice to people hiring an SEO is thin on this point. Its page on hiring an SEO says to ask for examples of previous work and success stories, and to check references with past clients.

It doesn’t say what a good case study contains. That gap is why this checklist exists.

Our view is that six elements separate a case study from an advert, and a missing one is a fair question.

1. A baseline in raw numbers. “Up 200%” means nothing until you know it started from 50 clicks.
2. Dated windows. Month and year at both ends, with the same length of time on each side.
3. A named data source. Search Console clicks are a different thing from a third-party “traffic estimate.”
4. Scope of work. What changed on the site, and what else was happening at the same time.
5. A business outcome, or an honest note that it’s withheld. Calls, booked jobs and revenue are the point of SEO, and clients often won’t share them.
6. Limits. A page that lists what it doesn’t show is more credible than one that claims everything.

The picture below turns the six elements into a quick checklist.

• Baseline in raw numbersStarting figures, not only a percentage.
• Dated windowsMonth and year at both ends, same length each side.
• Named data sourceSearch Console or GA4, not a vague traffic estimate.
• Scope of workWhat changed, and what else happened at the time.
• Business outcomeCalls, jobs or revenue, or a note that it is withheld.
• LimitsA list of what the page does not show.

Why do inflated SEO case studies exist at all?

Mostly because nobody checks. There’s no standard format, no audit and no cost to a flattering one.

Agencies pick their best account and the kindest date range, then present it as typical. That’s closer to selective storytelling than fraud, and it still misleads.

What should you ask for?

On your side, ask for the other end of the distribution. Ask which client in the last two years didn’t work out, and what the numbers looked like. An agency with only winners to show has either a very short history or a very selective one.

What about outright fabrication?

Outright fabrication is a separate problem. In August 2024 the Federal Trade Commission announced a final rule banning fake reviews and testimonials. The announcement says it prohibits creating or selling fake reviews and testimonials, covers AI-generated ones, and lets the agency seek civil penalties against knowing violators.

The rule covers reviews and testimonials, not traffic charts, but a made-up client quote next to a real-looking graph is the combination to distrust. Whether a given page breaks the rule is a question for a lawyer.

The more common problem is quieter: the numbers are real and the framing isn’t. That’s harder to catch, so most of this post is about it.

What are the red flags in an SEO case study?

The red flags are patterns, not proof. One of them can have an innocent explanation.

Three or more together should change how much weight you give the page. Each one below comes with the question to ask and when it’s fine to let it go.

Percentages with no baseline

A site that goes from 40 to 160 monthly clicks is “up 300%” and has gained 120 clicks. That’s a fine start for a new site and says nothing about leads.

Ask for both numbers. It’s fine if raw figures sit on the next line, and a flag if the percentage stands alone.

Undated or mismatched windows

“Results in 6 months” with no month named could be any year. Comparing a seasonal peak with a seasonal trough also flatters the result. Google’s guidance on debugging traffic drops names seasonality as a cause of traffic change and suggests comparing year over year, so ask for that view.

Impressions or rankings as the headline

Impressions count appearances, not visits. Google’s Search Console help defines clicks as the number of times a user clicked your site from Google Search results. It defines impressions as how many times your site appeared.

A case study that leads with impressions and buries clicks deserves a follow-up question.

Ranking screenshots have a similar problem. A first-place ranking for the client’s own company name says nothing about SEO. Ask which keywords the screenshot covers and whether any are branded.

No split between branded and non-branded

Searches for the client’s own name often grow because of PR, ads or word of mouth, not SEO. Google’s performance report documentation describes a branded and non-branded queries filter with a 16-month history starting from its introduction in March 2025. Ask whether the lift survives once branded queries are excluded.

Our post on growing branded search explains how to separate the two.

A core update in the window

Google says core updates are broad changes that don’t target specific sites, and that there are several a year. Its core update documentation tells site owners to confirm dates on the Search Status Dashboard. If a jump lines up with one, the agency’s work may not be the cause.

Cropped screenshots

A screenshot with no date range, no filters and no axis labels can’t be interpreted. Search Console also reports in Pacific Time, and it omits some anonymized queries from table rows, according to Google’s performance data documentation.

So totals and rows can differ. Ask for the full screen.

No scope of work

Say the agency rewrote the site, changed the CMS, ran paid campaigns and launched a PR push in the same period. One chart can’t be credited to one cause. The question is simple: what else changed that quarter?

Revenue claims with no tracking story

“Generated $400K in pipeline” needs a sentence on how: call tracking, form attribution, CRM matching. Without it, the number is a guess dressed as a result. Our post on SEO attribution explains why that chain is harder than it looks.

Guarantees attached to the case study

“We did it for them and will do it for you” is a promise, not evidence. Google says plainly that no one can guarantee a #1 ranking. Our post on SEO guarantees covers what to ask for instead.

Proof from a different business

A SaaS result doesn’t predict how a plumber’s service pages will do. Same industry, similar size and similar competition is the standard we hold ourselves to, and the reason we say when ours doesn’t match.

The case study verification checklist

Use this table when a proposal arrives with case studies attached. Print it, or paste it into an email to the agency.

You’ll rarely get a perfect score, since a small agency’s client may forbid sharing revenue. Weight the baseline, window and scope rows most heavily, because inflation depends on those.

We score it like this, and it’s judgment, not a standard. Count the passes:

• With six or more, including baseline, window and scope, go to the next step.
• With three to five, send questions and wait for answers.
• With two or fewer, treat the page as an advert.

How do you verify a case study yourself?

You can’t open someone else’s Search Console, but you can still test the claim in about fifteen minutes. Combine what’s public with a few questions an honest agency will answer without fuss.

1. Visit the client site. If the case study names it, check that it exists and looks like a real operating business. Check that the pages the agency says it worked on are live.
2. Search the client’s main service and city. See whether the site appears. This is a spot check, not proof, since results vary by searcher and location.
3. Ask for a live screen share. With the client’s permission, the agency opens Search Console and shows the date range, filters and Performance report. Google’s report documentation lists the fields you should see.
4. Request the comparison view. A year-over-year comparison and a 16-month view put a “growth” claim in seasonal context.
5. Ask for a reference call. Google’s guidance suggests checking references with past clients, and a 15-minute call tells you more than any slide.
6. Ask what went wrong. Every real engagement has a setback. An agency that can’t describe one is either editing its history or hasn’t done enough work.

1. Visit the client siteCheck that it exists and that the pages the agency worked on are live.
2. Search the main service and citySee whether the site appears. This is a spot check, not proof.
3. Ask for a live screen shareThe agency opens Search Console and shows the date range and filters.
4. Request the comparison viewA year-over-year view puts a growth claim in seasonal context.
5. Ask for a reference callA short call with a past client tells you more than any slide.
6. Ask what went wrongEvery real engagement has a setback.

If an agency refuses every step, you have your answer. If it declines the live view for confidentiality but offers a reference call, that’s a fair trade.

What to look at during the screen share

Once the Performance report is open, check these five things in order, and ask the agency to narrate as they go:

1. The date range matches the case study.
2. The search type is stated, because Google reports Web, Image, Video and News separately.
3. Both the clicks line and the impressions line show, so you can see which one the slide chose.
4. The Pages tab lists the specific pages the agency worked on, not just the site total.
5. The compare option, switched on for the same months a year earlier. If the growth is real and not seasonal, the gap will show.

A message you can send the agency

Paste this into an email and adjust the names. It asks for most rows of the checklist in one request.

“Before we decide, could you walk us through one case study? We’d like the raw start and end numbers with the months and the data source.”

“We’d also like the branded and non-branded split, what you changed and what else happened. Please show us either a live view of the Search Console data or a reference call.”

How an agency responds is part of the evidence. Defensiveness tells you something. So does a plain “we don’t have that number.”

Then ask what the first 90 days look like, using our post on the first 90 days with an SEO agency as a yardstick. Pair it with questions a good agency asks you, since a firm that interrogates its own proof and yours usually does careful work.

What does a hypothetical case study look like when you read it carefully?

Take some invented numbers. A plumber in Phoenix with three service pages sees this on an agency slide: “Organic traffic up 312% in six months.”

The arithmetic: if the site went from 100 clicks a month to 412, the gain is 312 clicks, a 312% increase. That sounds large and is small in absolute terms.

What changes when you add the questions?

Now add the questions. Suppose 40 of the original 100 clicks were searches for the company name, leaving 60 non-branded. Suppose 250 of the later 412 are branded, which a van-wrap campaign or a radio spot could cause.

Non-branded clicks are then 162. That’s a 170% rise (162 minus 60, divided by 60). It means about 100 more clicks a month from people who found the business without already knowing it.

The table and chart below show the same made-up numbers. Their figures are hypothetical.

• Before, all clicks100
• Before, non-branded60
• After, all clicks412
• After, non-branded162

Source: Hypothetical example

View the data as a table

Item	Value
Before, all clicks	100
Before, non-branded	60
After, all clicks	412
After, non-branded	162

Then ask about outcomes. A hundred extra non-branded clicks might produce several calls a month, or none, depending on the page. The slide can’t tell you.

Nothing in that example is dishonest. The slide just answers a smaller question than the one you’re asking, which is the usual shape of a misleading case study.

How do we present our own case studies?

We try to meet the checklist above, and we don’t claim a perfect score. Our case study pages report Google Search Console figures with the month and year, and they name the industry and country.

For example, our solar case study reports 24,216 Google clicks in September 2026 against 1,799 in April 2026. It also reports 5.4 times the clicks over the 90 days to 23 September 2026 compared with the prior 90 (65,688 vs 12,154). All figures come from Search Console.

That’s a solar company in India, so it says little about a US plumber. We have no US home-services SEO case study yet, and we don’t publish lead or revenue figures for any client because those are withheld. You can see the full set on our case studies page.

We’d rather you see the gaps than guess at them. Our guide to reading an SEO proposal applies the same test to the rest of the pitch.

What does the data not show about case-study checking?

A checklist improves your odds, but it won’t give you certainty, and these are the limits we know of.

• Verification from outside is partial. Without access to the client’s Search Console, you’re relying on screen shares, references and consistency. A determined fabricator can fake a screen share.
• Honest cases can look weak. A real result of 40 to 160 clicks in a quiet market may be a good job. It will lose to an inflated slide in a side-by-side comparison.
• Causation is hard. Even a clean case study can’t rule out other causes. Google’s core update documentation notes that changes can take days to months to show, which makes attribution to one action difficult.
• A case study is a sample of one. It shows method and honesty, not what your site will do.
• This is our view. The six elements, the red flags and the scoring rule are practitioner judgment. There’s no industry standard, and we have no study showing that following them predicts agency quality.

Other buyer-side questions deserve the same scrutiny. One is whether the agency does the work itself, covered in is my SEO agency outsourcing. Another is what the contract commits it to, covered in SEO contract terms.

When are we not the right fit?

A fair check of an agency’s proof includes the seller’s own limits, so here are ours. We’re probably not the right choice if any of these describe you.

• You want a ranking or a date promised in writing. We don’t sell ranking promises, and our SEO guarantees post explains why. If a competitor offers one, ours isn’t the proposal for you.
• You need proof from your exact trade right now. For US home-services businesses we have no same-trade case study yet. If that’s a hard requirement, wait or choose someone who has one.
• You need results inside 90 days. In our experience, steady organic leads take 6 to 12 months or more, so paid search is the faster lever. Our post on when an agency should show calls sets expectations.
• You only need a one-off fix. A single technical problem, or a site that’s basically sound, may not need an ongoing engagement.
• Your market is too small. If few people search for what you sell, SEO may not repay its cost. The SEO ROI calculator lets you test that with your own numbers first.

If you do want help judging the work, our SEO service page describes what an engagement includes. The SEO strategy guide shows the order we work in.

FAQ

If you want a second opinion on an agency’s proof, test your numbers with the SEO ROI calculator first, then Get Your Organic Growth Plan.

Sources

• Google Search Central, “Do you need an SEO?” (hiring an SEO): https://developers.google.com/search/docs/fundamentals/do-i-need-seo, accessed 22 Oct 2026
• Google Search Console Help, “Performance report (Search results)”: https://support.google.com/webmasters/answer/7576553, accessed 22 Oct 2026
• Google Search Console Help, “Performance report (Search results): Dimensions and data groupings”: https://support.google.com/webmasters/answer/17011259, accessed 22 Oct 2026
• Google Search Central, “Debugging drops in Google Search traffic”: https://developers.google.com/search/docs/monitor-debug/debugging-search-traffic-drops, accessed 22 Oct 2026
• Google Search Central, “Core updates”: https://developers.google.com/search/docs/appearance/core-updates, accessed 22 Oct 2026
• Federal Trade Commission, “Federal Trade Commission announces final rule banning fake reviews and testimonials” (14 Aug 2024): https://www.ftc.gov/news-events/news/press-releases/2024/08/federal-trade-commission-announces-final-rule-banning-fake-reviews-testimonials, accessed 22 Oct 2026