Abstract:Numerous studies and anecdotes demonstrate the "wisdom of the crowd," the surprising accuracy of a group's aggregated judgments. Less is known, however, about the generality of crowd wisdom. For example, are crowds wise even if their members have systematic judgmental biases, or can influence each other before members render their judgments? If so, are there situations in which we can expect a crowd to be less accurate than skilled individuals? We provide a precise but general definition of crowd wisdom: A crowd is wise if a linear aggregate, for example a mean, of its members' judgments is closer to the target value than a randomly, but not necessarily uniformly, sampled member of the crowd. Building on this definition, we develop a theoretical framework for examining, a priori, when and to what degree a crowd will be wise. We systematically investigate the boundary conditions for crowd wisdom within this framework and determine conditions under which the accuracy advantage for crowds is maximized. Our results demonstrate that crowd wisdom is highly robust: Even if judgments are biased and correlated, one would need to nearly deterministically select only a highly skilled judge before an individual's judgment could be expected to be more accurate than a simple averaging of the crowd. Our results also provide an accuracy rationale behind the need for diversity of judgments among group members. Contrary to folk explanations of crowd wisdom which hold that judgments should ideally be independent so that errors cancel out, we find that crowd wisdom is maximized when judgments systematically differ as much as possible. We re-analyze data from two published studies that confirm our theoretical results.

Using Cognitive Models to Improve the Wisdom of the Crowd

Metawisdom of the Crowd: How Choice Within Aided Decision Making Can Make Crowd Wisdom Robust

When is a crowd wise?

Wisdom of the crowd from unsupervised dimension reduction

Enhanced or distorted wisdom of crowds? An agent-based model of opinion formation under social influence

When social influence promotes the wisdom of crowds

Wisdom of the Confident: Using Social Interactions to Eliminate the Bias in Wisdom of the Crowds

Network dynamics of social influence in the wisdom of crowds

Extracting the Wisdom of a Smaller Crowd Fromdependent Quantile Judgments

Learning From Aggregated Opinion

The Wisdom of Reluctant Crowds

Aggregating Diverse Evaluations in Group Decision Making: an Approach Based on Wisdom of Crowds

Learning to Predict the Wisdom of Crowds

Aggregated knowledge from a small number of debates outperforms the wisdom of large crowds

The ambiguous role of social influence on the wisdom of crowds: An analytic approach

Mitigating Cognitive Biases in Multi-Criteria Crowd Assessment

Elicitation and aggregation of multimodal estimates improve wisdom of crowd effects on ordering tasks

Stuck in the Wisdom of Crowds: Information, Knowledge, and Heuristics

Wisdom of the Silicon Crowd: LLM Ensemble Prediction Capabilities Rival Human Crowd Accuracy

A Crowd Wisdom Management Framework for Crowdsourcing Systems

The Wisdom and Persuadability of Threads