When many people vote, individual errors can cancel one another out. But does that always happen? And can a very large group of only moderately well-informed people outperform a small group of very good experts?
The answer is surprising: yes – under specific mathematical assumptions. Those assumptions matter so much that the result should not be turned into a simple political argument.
A simple model
Assume that every decision-maker answers a factual question for which there is a verifiably correct answer. Let the probability that any one person is correct be p.
With three independent decision-makers, the majority is correct if exactly two or all three are correct:
The binomial coefficient 32 = 3 is read “3 choose 2”: there are three possible choices for which two of the three people are correct. The factor p2 represents two correct decisions, while (1−p) represents one incorrect decision.
For an odd number n of independent decision-makers, the general expression is:
Here nk is the binomial coefficient “n choose k”: the number of ways in which exactly k correct decisions can occur among n decision-makers.
The threshold is 50 percent
- If p > 0.5, majority voting becomes more reliable as the group grows.
- If p = 0.5, a larger group brings no advantage.
- If p < 0.5, a larger majority becomes increasingly reliable at being wrong.
A thought experiment: 11 scientists versus 1,001 citizens
Suppose, purely hypothetically, that 11 scientists each have an individual probability of pE = 0.80 of answering a clearly defined factual question correctly.
Under the idealized assumption of independent errors, the probability that the majority of these 11 experts is correct is about 98.83 percent.
The result is only about pV = 0.5358, or 53.58 percent individual accuracy.
This initially looks paradoxical. The individual citizen is only slightly better than chance. But if a very large number of such decisions are genuinely independent, it becomes extremely unlikely that more than half are wrong at the same time. This idea is closely related to the Condorcet jury theorem.
Turning the question around
Now fix the citizens’ individual probability at pV = 0.55. How many citizens are needed to match expert groups whose members each have pE = 0.80?
| Expert group | Citizens needed at p = 0.55 |
|---|---|
| 3 experts | 159 |
| 5 experts | 247 |
| 11 experts | 513 |
| 21 experts | 957 |
The crucial catch: independence
The mathematics is sound, but its most important assumption is also the most problematic: the decision-makers’ errors must be sufficiently independent.
If 1,000 people rely on the same false source, we do not have 1,000 independent judgments. Shared media, social networks, political campaigns, rumors and cultural assumptions can strongly correlate errors.
The same issue applies to expert groups. Scientists may read the same publications, use similar models, or share systematic measurement errors.
Factual questions are not value questions
The model assumes that there is an objectively verifiable correct answer. That is at least meaningful in principle for many scientific and technical questions.
Political decisions, however, often contain value judgments as well. Science can investigate likely consequences, risks and costs, but it cannot by itself determine which goals a society should pursue or how competing interests should be weighted.
The thought experiment therefore shows neither that referendums are superior to experts nor that experts should replace democratic decisions.
Back to Random Forests
The starting point for this thought experiment was a machine-learning method: the Random Forest. It combines many decision trees. The trees should not merely be reasonably good; through bootstrap samples and random feature selection they should also make different errors.
Three factors interact in successful ensembles: individual quality, group size, and diversity of errors.