How to pretest a survey before you pay for participants

August 7, 2026 · 4 min read

Most wasted participant budget isn't wasted on bad ideas — it's wasted on good ideas that were fielded before the survey itself was ready. A pretest is the cheapest insurance you can buy. Here's what to check before a single real participant sees your study.

1. Walk every branch

Skip logic and randomized conditions are where surveys quietly break. Confirm that every arm routes correctly, that no one gets stranded, and that embedded data carries through to the end. Bugs here silently corrupt your data rather than throwing an error.

2. Check that your manipulation actually moves something

The most expensive failure is a manipulation that reads clearly to you but does nothing to respondents. Look at whether your conditions produce a difference on the manipulation checkbeforeyou care about the outcome. If the manipulation doesn't move, nothing downstream will.

3. Read your wording like a stranger

• Double-barreled questions (“how clear and useful was this?”).
• Leading or loaded phrasing that telegraphs the “right” answer.
• Scales with missing or overlapping options.
• Jargon that means something to you but not to your sample.

4. Estimate the effect, not just the plumbing

A pretest can also give you a rough sense of direction and size — enough to decide whether the effect is worth powering for. AI respondents are useful here: if a real effect shows up in a cheap simulated run, that's a green light to invest in a proper sample.

5. Then field with confidence

Once the logic is clean, the manipulation moves, and the wording is tight, you can spend on human participants knowing you're paying to measure an effect — not to debug a survey.