Overconfidence in judgment for repeatable events.

Abstract

People are overconfident in their judgments about repeatable events. As an example, suppose respondents are instructed to diagnose each of a series of hypothetical patients as having one of two diseases in the following two-stage procedure. For each patient, the respondent first indicates what disease they believe is afflicting the patient. Then the respondent states a 50% to 100% probability judgment that the patient actually has the indicated disease. Over a series of such trials, the average of the probability judgments tends to exceed the proportion of correct diagnoses (i.e. respondents tend to be overconfident). A neural network-based probability judgment (NBPJ) model and an exemplar-based probability judgment (EBPJ) model were developed for these kinds of tasks, and accounts for overconfidence were derived from each. The NBPJ asserts that people's learning of ecological probabilities is essentially veridical. However, their classification responses are fundamentally probabilistic, which results in overconfidence. The EBPJ proposes that people learn by storing past examples, and that their judgments are often based on the first example they happen to retrieve. In this model, reliance on small samples of exemplars in judgment leads to overconfidence. The models' accounts of overconfidence were compared in three experiments. The first of two key results was that eliminating the choice stage of the judgment process and directly reporting on the probability of one disease led to an increase in overconfidence. This is anticipated by the EBPJ because this procedure encourages retrieval of less information than the two-stage procedure. The NBPJ predicts the opposite result, because it locates overconfidence at the choice stage, which is eliminated in a single-stage, direct report procedure. The second principal result was that an instruction to retrieve many exemplars reduced overconfidence. This result directly supported the EBPJ's account, but was not expected by the NBPJ. Implications for current theories of likelihood judgment are discussed.