Components of probability judgment accuracy: Individual consistency and effects of subject matter and assessment method

Abstract

An experiment is reported in which subjects assigned probabilities to the outcomes of basketball games and to the truth of general-knowledge items. Three different methods were used for eliciting subjects' probability judgments. Subjects were more successful in selecting answers to the general-knowledge questions than they were in picking basketball game winners. The overall accuracy of their probability judgments for general-knowledge items was superior, too. On the other hand, subjects' judgments about general-knowledge questions were more overconfident, more poorly calibrated, and included greater scatter. One method of probability assessment gave subjects an irrelevant cue. This was found to increase confidence and overconfidence and to hurt calibration. Correlations between measures of performance on general-knowledge questions and basketball predictions showed substantial individual consistency in confidence, but only weak consistency in other components of judgment quality. Theoretical and practical implications are discussed.