How many people you need for a usability test: five, twenty or forty
Everyone who has run a test knows the five-user rule. It answers only one question, though. When five people are enough, when you need forty, and how to tell you have enough.
The Tack team··7 min read
Short answer
It depends on what you want to learn. To find usability problems, about five people per round are enough. They uncover around 85% of the problems that at least a third of users run into. Three rounds of five with fixes in between beat one round of fifteen.
For numbers, such as a task success rate or an average difficulty score, you need more people. Twenty give you a rough estimate; Nielsen Norman Group recommends forty for quantitative studies. To compare two versions you need more still, because both numbers carry uncertainty.
On this page
Two different questions, two different numbers
“How many people do I need” has two answers, because a test can look for two different things:
- What gets in people’s way. You want to find problems and fix them. Whether 30% or 40% of people hit a problem does not matter; that it exists and what causes it does. Few people are enough here.
- How well it works. You want a number that holds up: a task success rate, a difficulty score, a comparison with the last version. Precision decides here, and it grows slowly with the number of people.
Most small teams mainly need the first. Numbers come later.
Where the five-user rule comes from
In 1993 Jakob Nielsen and Tom Landauer found that one user runs into 31% of an interface’s problems on average. Each further user finds some new ones and some you already know. The share of problems found grows by the formula 1 − (1 − p)n, where p is the probability a person hits a problem and n is the number of people (NN/g, MeasuringU).
| People | A problem 31% of people hit | A problem 10% of people hit |
|---|---|---|
| 1 | 31% | 10% |
| 3 | 67% | 27% |
| 5 | 84% | 41% |
| 10 | 98% | 65% |
| 20 | 100% | 88% |
The table also shows when the rule does not hold. Five people reliably find common problems, but rarer ones that one user in ten hits, they find only about 40% of the time. When you have many kinds of users or look for rarer issues, you need more people.
A 2003 study by Laura Faulkner shows the same. She tested 60 people and then drew random groups of five. Some groups of five found 99% of the problems, others only 55%. Groups of ten always found at least 80%, and groups of twenty at least 95% (Behavior Research Methods).
Why several rounds of five, not one big one
When you have a budget for fifteen people, do not put them all in one round. Split them into three:
- The first five find the most common problems. You fix them.
- The second five confirm the fixes work and hit problems the first ones were hiding. A person stuck at sign-up never reached the rest of the app.
- The third five confirm you are done, or find the last leftovers.
With a prototype built with AI this pays off even more, because a fix between rounds takes minutes. You can run three rounds in a few days.
How many people you need for numbers
When four people out of five complete a task, the success rate is 80%. But the true rate among all users could be anywhere between 36% and 98%. The interval narrows as you add people, but slowly:
| People | Succeeded | True success rate is likely between |
|---|---|---|
| 5 | 4 | 36% and 98% |
| 10 | 8 | 48% and 95% |
| 20 | 16 | 58% and 93% |
| 40 | 32 | 65% and 90% |
| 100 | 80 | 71% and 87% |
In practice:
- Five people tell you whether a task works or is broken. Do not put a success rate from five people in a slide deck.
- Twenty people give a rough estimate, useful for a team decision.
- Forty people is what Nielsen Norman Group recommends for most quantitative studies (NN/g).
Work out the exact interval for your own results in the calculators on the metric pages, such as SEQ or CSAT.
Comparing two versions
Want to know whether the new version beats the old one? You need more people than for a single measurement, because both numbers carry uncertainty and the difference has to be bigger than both. With five people per version you only see large differences, typically when one version works and the other does not.
For a small team it is usually more sensible to skip significance testing and test in rounds: when a problem from the first round is gone in the second, the fix worked.
Quick summary
| What you want to learn | How many people |
|---|---|
| What gets in people’s way most | 5 per round, ideally 2–3 rounds |
| Did the fix work? | 5 in the next round |
| A rough success or difficulty estimate | 20 |
| A number that holds up in a report | 40 |
| A difference between two versions | Dozens per version or more |
FAQ
How many participants do I need for a usability test?
To find problems, about five people per round are enough, ideally in two or three rounds with fixes in between. When you need reliable numbers, such as a task success rate, plan for twenty to forty people.
Does the five-user rule always hold?
No. Five people find around 85% of the problems that at least a third of users hit. Rarer problems, ones that one user in ten hits, they find only about 40% of the time. The rule also does not hold for numbers such as a task success rate, where five people are too few for a reliable estimate.
Why are several small rounds better than one big one?
The first round finds the most common problems, you fix them, and the next round hits problems the first ones were hiding. It also confirms the fixes work. Three rounds of five give you more than one round of fifteen.
How many people do I need for a quantitative usability test?
Nielsen Norman Group recommends 40 people for most quantitative studies. Twenty give you a rough estimate. Work out the confidence interval for your own results in a metric calculator.