How to write a case study with evidence, not inflated claims
Neaptide · September 6, 2026 · 7 min read · Updated: October 4, 2026
Write a case study with verifiable evidence: Vodafone’s A/B test, relative changes, missing data, source attribution and an interview brief.
On this page
Vodafone’s case on web.dev reports 8% more sales alongside the conditions of its A/B test. Together they tell the reader more than a percentage headline: what changed, what was compared and what the evidence supports. Use the same discipline when writing your own case.
What a case study should explain
A case study examines a specific project: the problem, constraints, decisions and observed outcome. A prospective client reads it to judge whether the approach would suit their own situation, and a list of tools will not tell them.
Gather the evidence before writing the headline
- Define who had the problem, when it occurred and why it mattered.
- Collect the brief, earlier versions, change dates and reports.
- Separate your contribution from the client’s work and other suppliers’ actions.
- Check metric definitions, periods, filters and tracking changes.
- Record the result and what you cannot yet establish.
If a number has no source, ask the person who owns the data. Do not ask AI to fill the gap with something plausible. A draft can carry an open question. A published claim cannot.
Vodafone’s real case: what the published data supports
Vodafone’s case on web.dev, published on 17 March 2021, compares two landing pages in an A/B test with a 50/50 traffic split. Design and functionality were identical; one version received Web Vitals optimizations. This is Vodafone’s published result, not a Neaptide project.
| Metric | Baseline version | Optimized version | Reported change |
|---|---|---|---|
| LCP | 8.3 s | 5.7 s | 31% improvement |
| Sales | Not disclosed | Not disclosed | +8% |
| Lead-to-visit rate | Not disclosed | Not disclosed | +15% relative |
Using the rounded LCP values, (8.3 − 5.7) / 8.3 ≈ 31.3%, consistent with the reported 31%. The lead-to-visit increase is 15% relative. The baseline conversion rate is not disclosed, so the percentage-point change cannot be calculated.
The A/B test supports a comparison within this experiment. It does not promise other websites 8% more sales. The case explicitly withholds absolute business metrics; do not reconstruct lead counts or revenue from the percentages.
Keep metrics from different systems distinct
Search Console clicks measure clicks from Google Search, impressions measure appearances in results, and CTR is clicks divided by impressions. None of these is a lead count. For enquiries, use a source that records them, such as validated form submissions or CRM data.
State the source, dates, segment and counting rule beside each table. If tracking or spam filtering changed, say which figures can still be compared. A dashboard screenshot without that context does nothing for a weak claim.
Show decisions, not a task list
Choose two or three decisions that shaped the project. For each, explain what blocked the user, which options the team considered, why it chose one and how it checked the result. Ask a participant: “What did you learn that made you change the original plan?” If the records support the answer, include it. It says more about the work than “we analysed, built and launched”, and it lets a prospective client compare your conditions with their own.
One order that works: the problem and constraints, the starting state, the decisions and the reasons for them, validation, the outcome and its limitations, then a lesson for similar projects. If you have no business metrics, show an outcome you can verify at another level, such as a released feature, a fixed defect or an acceptance scenario that passed. Say plainly that it is not evidence of revenue growth.
Choose illustrations that support the argument
- Before-and-after screenshots at the same scale, explaining the specific change.
- A chart with units, periods and underlying values, plus an accessible text table.
- A document or interface excerpt that supports an important decision.
An illustrated cover is fine. An AI-generated image is no substitute for a real analytics screen or a customer testimonial. Agree with the people involved which material can be published. Remove personal information that is not needed, but keep the context a reader needs to understand the measurement.
Interview questions and the final check
The interview brief covers the problem, constraints, decisions, measurement and limitations. Write only from answers you can back with evidence. Before publishing, check that the headline claims no more than the evidence shows, then go through the headings, captions, dates and links to related services. Keep the measurement period and the limitations next to the result, so that they travel with it when someone quotes it. Do not promise another client the same percentage gain.