The Perceived and Actual Influence of Social Diversity on "Crowd" Judgment Accuracy

Abstract

Statistically aggregating even a few estimates can yield improvements over individual judgments. But before aggregating, one must first decide whom to ask. One strategy might be to ask a socially diverse group, as diverse people are expected to contribute different perspectives. Related to this idea, the aims of this dissertation were threefold. The first aim was to determine the necessary conditions under which socially diverse crowds outperform homogeneous crowds for a given numerical judgment. The second aim was to test to what extent these conditions are met in real judgment tasks across a variety of social identities and judgment questions. The third aim was to determine how realistic laypeople are when appraising the accuracy advantages – or lack thereof – of socially diverse crowds. Chapter I reviews relevant literature on judgment, groups, and diversity. The discussion makes clear that the question of whether social diversity boosts crowd accuracy cannot be answered by the extensive literature on group diversity. The chapter nevertheless reviews that work and uses it to guide hypotheses while at the same time drawing distinctions between the questions asked here and the questions previously answered. A model in Chapter II tests when socially diverse crowds will outperform homogeneous ones. Findings suggest that diversity only improves crowd judgment when the relationship between group membership and judgment is at least moderate in size and when the true value lies between the distributions of the two groups. Chapter III then seeks to observe these conditions in real judgment tasks. Results indicate that the conditions for diversity benefits are rarely observed in empirical data. People’s social identities did not strongly bias their judgments across a wide variety of topics, and homogeneous and diverse crowds performed about equally well on numerical judgment tasks. Studies IV.1-IV.5 examined lay beliefs about diversity advantages. People typically overestimated the relative accuracy of socially diverse groups over homogeneous groups. That overestimation arose from people’s erroneous assumptions about meeting the “diversity benefit conditions” in the model (Studies IV.1 – IV.2). Specifically, people (1) overestimated the effect of social identity on judgment and (2) expected bracketing to occur to some degree. People expected diverse crowds to be optimal when they assumed these conditions are present, but not when they assumed that those conditions were absent. Studies IV.3-IV.5 suggest that people are willing to act on those erroneous beliefs. They are far more likely to choose advice from a diverse crowd, even when a homogeneous crowd would be more accurate (Study IV.3), and they are willing to pay more money to see advice from a diverse crowd than a homogeneous one when completing a numerical judgment task (Study IV.4-IV.5). Finally, Chapter V reviews the findings in the dissertation, discusses implications and limitations of the present work, and proposes future studies to address remaining questions.