A validation survey counts how common a problem is among people who fit your customer. Screen out everyone else first, ask about the last month rather than "usually", avoid agree-or-disagree questions, and keep it under ten questions. About 100 qualified answers give a usable estimate.
This is for founders who have done a few interviews and want to know whether what they heard is common. The running example is SafePlate, an imagined meal planning app for families with food allergies.
In short:
- Interviews find the problems. Surveys measure how widespread they are.
- Screening questions keep non-customers out of your numbers.
- Agree-or-disagree questions push people towards "agree".
- 100 answers give a margin of about 10 points either way.
Surveys count what interviews found
A closed-ended survey can't discover a problem you didn't think to ask about; open-ended questions can reveal others. It can tell you whether a problem from six interviews applies to 10% or 60% of your market.
So run customer interviews first. Use the survey to test the two or three problems you heard most, in the words people used.
Screen out people who aren't your customer
Start with two or three screening questions. Anyone who doesn't fit goes straight to a thank-you page, and their answers don't count.
For SafePlate:
- Ask "Does anyone in your household have a food allergy?" (Yes, No)
- Ask "Who plans most of your household's meals?" (Me, Someone else, We share it)
- End the survey for "No" or "Someone else".
Without screening, the results mix allergy families with everyone else, and the problem looks rarer than it is.
Ask about the last month, not "usually"
People report what they did recently more accurately than what they "usually" do. Give a clear timeframe in every behaviour question.
"In the last month, how many times did you check a product label for allergens while shopping?" is answerable. "How often do you check labels?" invites a guess.
The same rule as interviews applies. Ask about real past behaviour, and avoid "Would you use an app that...?" Answers to hypotheticals are optimistic.
Avoid three wording traps
Small wording choices change answers. Pew Research Center's guide to writing survey questions documents the main ones, including the tendency of some people to agree with any statement, called acquiescence bias.
Each trap has a better version: ask one specific thing in neutral words.
| Trap | Avoid | Better |
|---|---|---|
| Double-barrelled | "Is planning hard and slow?" | Two separate questions |
| Agree or disagree | "Planning is stressful. Agree?" | "How stressful was planning last week?" |
| Leading | "How much would an app help?" | "What do you use to plan now?" |
Also keep answer scales balanced, with as many negative as positive options. The Nielsen Norman Group compares common formats in its article on rating scales in UX research.
Use a nine-question template
This template keeps the survey under five minutes. Adapt each line to your app.
- Screen whether the person fits the customer (2 questions).
- Measure how often the problem happened in the last month.
- Measure how much time or money it cost last time.
- Map what they use now and whether they pay for it (2 questions).
- Rank which of your three problems is the worst for them.
- Ask them to describe the last time it went wrong, in one text box.
- Invite them to a 15-minute call, with an optional email field.
The open question gives you quotes for your landing page. The invitation fills your interview list, and people who leave an email are your warmest leads.
Collect about 100 qualified answers
The number of answers decides how precise your percentages are. For a result near 50%, the 95% margin of error is about 10 points with 100 answers, 7 points with 200, and 5 points with 400.
That's precise enough to tell "about half" from "about a fifth". It isn't enough to compare small groups inside the sample. Online calculators such as SurveyMonkey's sample size calculator do the maths for other cases.
Know who you're reaching
Where respondents come from shapes the result. Your own followers like you, and a parenting forum leans towards engaged parents. Note the source of each answer and compare them.
Paid panels such as Prolific recruit participants who match screening criteria. They cost money but reach people outside your circle.
Sort open answers with a fixed code list
Open answers are where the surprises are. Read the first 20 yourself and write five to seven codes, such as "label reading", "eating out", "school lunches", "cost of safe food".
Then give an AI assistant the full list of answers and your codes, and ask it to tag each answer with one code without inventing new ones. Spot-check ten. The same method works for reviews and Reddit threads.
Build the next step in GrowApps
If the survey shows a common, painful problem, the next test is a working version. GrowApps builds real apps for web, Android and iOS from a chat, and a GrowApps app has sign-in and saved data built in (sign-in, data, AI and payments).
Paste your top survey findings and two or three open-answer quotes into the first message, so the first version targets the most common problem.
Draft your survey in ten minutes
Open a free form tool such as Google Forms or Tally. Write your two screening questions and one past-month frequency question first. Send that three-question version to five people and check that they understand it. Then read how to validate an app idea in a weekend.
Frequently asked questions
Should I ask people what they would pay?
Price questions in a survey give a rough range at best. If you ask, use the four Van Westendorp questions on one product description. A real price on a landing page tells you more.
Should I offer a prize for answers?
A small prize lifts response rates but attracts people who just want the prize. Use screening questions and check open answers for nonsense before you count a response.
Sources and further reading
- Writing survey questions, Pew Research Center
- Rating scales in UX research, Nielsen Norman Group
- Qualitative surveys, Nielsen Norman Group
- Sample size calculator, SurveyMonkey
- Acquiescence bias, Wikipedia
- Prolific, research participant panel