Choose research by the question you cannot answer, not by the quickest tool or the report with the most charts. Existing feedback can reveal signals already present; interviews explore context and possible explanations; a survey can measure responses to defined questions within a specified audience. Each method has limits, and combining them does not automatically remove those limits.
Start with a decision: should we redesign a step, clarify its messages or investigate a different cause? Write what is known, what is assumed and what evidence would change the decision. That is a better basis for commissioning research than requesting a “comprehensive survey” before deciding what it should investigate.
Three methods, three kinds of evidence
| Method | Useful for | Main limitation | Required output |
|---|---|---|---|
| Existing feedback | Finding recurring support or comment themes | Excludes non-contacting users and may repeat the same customer | Category definitions, counting unit and source limits |
| Interviews | Understanding event sequences, constraints and language | Does not automatically measure prevalence | Case context with evidence and interpretation separated |
| Survey | Measuring answers to defined questions in a specified audience | Poor questions or sampling can produce misleading numbers | Questionnaire, recruitment method, denominators and exclusions |
GOV.UK connects the choice of research activity to what the team needs to learn. Research-planning guidance. You do not need all three methods in every project. Ask for a justified sequence with transition criteria rather than a bundle of available techniques.
Worked hypothetical example: subscription invoicing feedback
All counts are fictional. A team has 240 support messages in a defined period from 160 distinct accounts. Forty-eight messages concern invoicing, but they came from only 18 accounts. That is 20% of messages and 11.25% of accounts that contacted support. Neither is the percentage of all customers affected, because the total customer population and its coverage are not established in this analysis.
Before sending a survey titled “Why are invoices difficult?”, the team checks the classification. Some messages request a downloadable copy, some concern issue timing and others query an unexpected amount. Combining them may lead to redesigning an entire dashboard when one route needs a small clarification and another requires a different investigation.
The appropriate first step is to improve categories and connect messages to accounts and tasks, using authorized data with unnecessary personal information removed. Focused interviews with relevant cases can then explore what people were trying to accomplish. If a specific question emerges, such as whether account administrators know where to find a downloadable copy, a later study can be designed to measure that understanding or observe the task.
If a survey is selected, define who should respond: the person who actually handles invoices, not every user in an account. Multiple answers from one organization should not silently become independent customer records if the decision is at organization level. Choose the analysis unit before collection.
The numbers in this example are useful because they expose a denominator problem, not because they establish an industry benchmark. A decision maker should be able to reproduce both calculations and understand exactly which population each describes.
When should you start with existing material?
Start with available feedback when access is authorized, collection context is known and it addresses a clear question. Check period, channel, product version and segment. Separate requests for help, functional defects and feature suggestions. Customer silence is not proof of satisfaction; repeated messages from one account do not establish widespread prevalence.
User Research can structure that evidence and identify questions requiring relevant participants. If records are incomplete or their use is not permitted, do not upload them to another tool simply to speed classification. Discuss an authorized sample or alternative method with the data owner.
Existing evidence can also help decide whom not to recruit. For example, a participant who never uses the invoicing function may have useful opinions about the brand but cannot describe a recent invoicing task. Relevance to the decision matters more than willingness to answer.
When are interviews or a survey appropriate?
Use interviews when the event sequence, terminology or involved people are unclear. If the thing to measure is understood and an appropriate audience is accessible, investigate whether a survey fits. Sampling and response bias remain separate from response count: more answers do not repair the exclusion of an important group from invitations.
Pew Research Center explains that wording can affect survey responses. Question-writing reference. Test comprehension before distribution. Avoid one question about the “ease, speed and accuracy” of invoicing; respondents may judge each differently, leaving a single answer difficult to interpret.
If the question concerns market segments or alternatives outside the current user base, it may require Market & Offer Research. A survey of existing customers is not automatically a study of the whole market, even if many people respond.
Connect learning to a testable change
Understanding an obstacle does not establish that the proposed solution will remove it. Product Validation can test a specific assumption against an agreed decision rule. The result may be a content improvement, a changed step or no modification because the evidence does not support one.
Ask the supplier why the method was chosen, what it permits you to conclude, what it cannot establish and how access gaps affect the result. If the method changes during delivery, record the evidence that prompted the change rather than switching because early findings were disappointing.
Method-selection acceptance checks:
- Question, decision, audience and analysis unit are explicit.
- Records and participants are accessible with appropriate authorization.
- Duplicates, denominators and categories are documented.
- A survey is not called representative without a defensible sample design.
- Reported accounts are not treated automatically as observed performance or causal proof.
- Deliverables explain inference limits and the next decision.
Frequently asked questions
Is a survey always cheaper?
No. Question design, recruitment and analysis may exceed the work of a focused qualitative study. Compare the cost of suitable evidence, not merely the cost of distributing a link.
Are platform or store reviews enough?
They may reveal useful themes, but describe people who reviewed through that channel. They do not automatically reveal the views or experiences of non-reviewers.
Can the results be combined into one percentage?
Do not combine messages, interviews and survey responses with different denominators. Compare patterns while preserving each source and counting unit.
When should several methods be used?
When methods provide complementary evidence for the same decision, whether they address different questions or compare what different sources reveal about the same question. Plan how to connect findings and explain contradictions. The number of methods alone does not establish strength; clarify each method's contribution and limits.
Begin with the decision and available evidence, then commission a method that adds information you genuinely need.
