Research · Jun 25, 2026 · 10 min read · by the Pressfold team

Survey design for PR that holds up to scrutiny

There is a particular silence that follows a pitch built on a bad survey. The reporter does not argue. They do not ask follow-up questions. They simply do not reply, because they read the methodology line, saw a sample of 300 people recruited through a competition entry form, and decided the story was not worth the reputational risk of running. The survey was the whole campaign, and the survey was junk, and no amount of clever framing was ever going to fix that.

Survey-led PR works because original data is one of the few things a brand can offer a newsroom that the newsroom cannot easily get elsewhere. But that only holds if the survey is built to be defended. A reporter putting their name on your number is implicitly vouching for it, and the good ones have learned to spot the tells of a survey designed to produce a headline rather than to measure reality. This piece is about building the kind that survives scrutiny — from the journalist, from their editor, and from anyone who decides to check.

Start from the claim you want to be able to make

Good survey design runs backwards from publication. Before writing a single question, write the two or three findings you hope to be able to report — and then ask whether a fair survey could plausibly produce them. This sounds like cheating; it is the opposite. It forces you to confront, while you can still change the instrument, whether your questions are capable of supporting the story without being rigged toward it.

The distinction matters. Designing toward a hypothesis is legitimate research practice: you suspect renters are cutting back on insurance, so you ask about it directly and cleanly. Designing toward a predetermined answer is fabrication with extra steps: you write the question so that almost any respondent will tick the box you need. The first produces a finding you can stand behind. The second produces a number that collapses the moment anyone looks at how it was generated.

A practical safeguard is to draft the questions and the dream headlines in two separate columns, then hand them to a colleague who was not involved and ask whether the questions could just as easily have produced the opposite headline. If they could not, the questions are loaded, and a sharp reporter will see it too.

Sample size is necessary but it is not the whole story

The first thing a reporter checks is how many people you asked. There is no universal magic number, but there are thresholds below which serious outlets quietly stop taking you seriously. A nationally framed claim built on a couple of hundred responses invites the obvious question of margin of error, and the margin on small samples is wide enough to swallow most findings whole. Bigger is better up to a point, after which the gains shrink and the cost climbs.

But raw size is the least interesting part of the sample. Who those people are matters far more. A thousand respondents who all came from the same newsletter, the same incentive scheme, or the same narrow demographic are not a thousand independent observations of the public — they are one biased slice repeated a thousand times. Reporters covering a national story want to know the sample resembles the population the headline claims to describe. A survey that says "UK adults" but actually surveyed a self-selected online panel skewed young and urban is making a claim it cannot support, regardless of how many people answered.

This is also where subgroup claims get dangerous. The headline may be solid on the full sample, but the moment you start reporting that "42% of over-65s said X," you are slicing a thousand responses into a cell that might contain eighty people. The error bars on that cell are enormous. Reporters who know what they are doing will not run a subgroup figure built on a handful of responses, and you should not offer one.

If subgroup angles are central to the story you want to tell — regional differences, generational splits, men versus women — the time to address that is at the design stage, by boosting the sample so that each cell you intend to report carries enough responses to mean something. Deciding after the fact that you want a robust figure for a small group is too late; the responses either exist or they do not. Planning the sample around the cuts you plan to publish, rather than the headline alone, is the difference between a survey that supports a rich set of angles and one that can only carry a single national number safely.

Neutral questions are the heart of the matter

The most common way a PR survey turns to junk is the leading question. It is also the easiest mistake to commit without noticing, because a leading question often reads perfectly naturally to the person who wrote it. "How concerned are you about the rising cost of energy bills?" presupposes concern and presupposes a rise; the only thing left to measure is the intensity of a feeling you have already installed. The clean version asks whether the respondent is concerned at all, and offers a genuine route to "not concerned" without making it feel like the wrong answer.

Several patterns reliably contaminate results. Loaded adjectives that smuggle in a judgement. Double-barrelled questions that ask about two things at once, so you cannot tell which one the answer refers to. Unbalanced scales that offer four shades of agreement and one of disagreement. Questions that assume a behaviour the respondent may not engage in. And the absence of a "don't know" or "none of these" option, which forces people into an opinion they do not hold and inflates whatever you were measuring.

Order effects matter too. Ask an emotionally charged question first and it colours everything that follows. A well-built questionnaire puts neutral, factual questions before attitudinal ones, randomises answer options where appropriate, and resists the urge to "remind" respondents of context that nudges them toward a particular reply. The goal is a questionnaire that a hostile statistician could read without finding a thumb on the scale.

A quiet but powerful test is to pilot the questionnaire on a small group before fielding it in full, and to read each respondent's answers as a set rather than question by question. Pilots surface the questions people misread, the options nobody picks, and the scales that bunch everyone at one end. They also reveal the questions that produce a suspiciously lopsided result — which is sometimes a genuine finding and sometimes a sign the question was leading. Catching this before the full field stage is cheap; catching it after, when a reporter queries the result, is not. The few hundred you spend piloting protects the entire campaign that rests on the instrument.

Weighting fixes what recruitment cannot

Almost no survey produces a perfectly representative raw sample. Some groups respond more readily than others, and certain panels over-represent particular ages or regions. Weighting is the standard, legitimate correction: you compare your sample's composition to known population figures and adjust each respondent's contribution so the totals match reality. Done properly, it is not manipulation — it is the thing that lets you say "UK adults" honestly.

What matters for credibility is being transparent that you weighted, and on what. A method note that says the data was weighted to be nationally representative by age, gender, and region tells a reporter you took representativeness seriously. The danger sign is heavy weighting applied to a sample that was badly skewed to begin with, because weighting can only stretch the data so far. If your panel contained barely any respondents from a group that makes up a fifth of the population, no weighting scheme can responsibly inflate that handful to speak for all of them. Recruitment quality sets a ceiling that weighting cannot exceed.

There is also a presentation discipline that comes with weighting. Once data is weighted, the percentages you report are based on the weighted totals, but the base sizes you cite should reflect the actual number of people who answered. Mixing these up — quoting a weighted percentage against an unweighted base, or vice versa — is the kind of inconsistency a careful editor spots. Keep the two clearly separated in your tables, and when in doubt, report both the unweighted count and the weighted percentage so nobody has to guess what the figure rests on. Reporters reward this kind of clarity by trusting the rest of the dataset more readily.

The stats reporters reject, and why

Certain findings get spiked on sight by experienced journalists, and it is worth knowing the patterns so you do not build a campaign around one. Percentages with no base, where you report "73% agreed" without ever saying 73% of how many. Findings that are statistically indistinguishable from noise but are reported as meaningful differences. Correlations dressed up as causes, where two things move together and the press release implies one drives the other. And the perennial favourite, the survey that discovers, with suspicious convenience, exactly the thing the commissioning brand sells.

That last one deserves special caution. If a mattress company's survey reveals that the nation is dangerously sleep-deprived and would benefit from a better mattress, the reporter's guard goes straight up. The finding might even be true, but the conflict of interest makes it look manufactured. The way through is to let the data lead somewhere genuinely interesting rather than somewhere conveniently on-message, and to be upfront about who paid for the research. Disclosed commissioning is normal and accepted; disguised commissioning, discovered later, is the kind of thing that ends relationships with newsrooms.

A survey is only the raw material, of course — turning it into something publishable is a separate craft, and the structure that does it is covered in the anatomy of a data story journalists actually publish. The point of clean design is simply that it gives you raw material worth shaping.

Build the method note before you build the story

The discipline that ties all of this together is writing the methodology statement early, not as an afterthought. If you draft the method note while designing the survey — sample source, size, fieldwork dates, weighting approach, who commissioned it — you are forced to confront every weakness while you can still fix it. A method note that is embarrassing to write is a survey that needs redesigning before it goes into the field, not after.

When the method note is clean, everything downstream gets easier. Outreach becomes a matter of presenting a finding you can defend rather than hoping nobody asks how you got it. Reporters move faster because the answer to their first question is already in front of them. And the story, once it runs, holds up if a competitor or a fact-checker decides to poke at it. The unglamorous truth of survey-led PR is that the credibility is built at the design stage, long before anyone writes a headline — and the questions covered here are most of what separates data a serious outlet will print from data it will quietly ignore.