Dispersion of Values: A Discursive Examination of Attitude Self-report Item Designs

Abstract

Recent scholarship has found that social and political inequality can be reproduced in psychological researchers’ development of study designs and materials (McClelland et al., 2020; Westbrook & Saperstein, 2015). One place where this reproduction occurs may be psychological research that employs self-report methods of data collection. The aims of the research were to: 1) examine the self-report items developed and administered in two areas of psychological research, including public opinion on abortion in the U.S. and women’s body esteem self-attitudes; 2) examine whether these self-report items have been designed with conceptual frames, wording, logics, and imagery that contain biased information about abortion and women, respectively; and 3) to consider how this biased information might have made its way, unnoticed, into researchers’ design choices. Study 1 focused on the self-report items developed to produce knowledge about the U.S. public’s attitudes towards abortion. This study employed a systematic review methodology to gather all unique self-report items that were asked about attitudes towards abortion in the U.S. from 2008-2020. These items were drawn from 92 studies including public opinion polling and social science research about abortion attitudes that is widely disseminated to the public. A discursive examination of these items identified that the items contained biased information about abortion as inherently immoral and harmful, and positioned respondents to support further restrictions on abortion. These findings suggest that efforts to document the U.S. public’s attitudes towards abortion have relied on biased study materials, and that disseminating these findings likely encourages further restrictive attitudes amongst the U.S. public and policymakers who legislate abortion access. Study 2 focused on a set of self-report items developed to measure women’s body-esteem in psychological research studies. This study employed a cognitive debriefing methodology to invite a sample of women diverse in racial/ethnic and sexual identities (N=411) to respond to four self-report items that asked about being visually evaluated by other people in public settings (e.g., “Other people consider me good looking”). Participants were subsequently asked to respond to the following question for each of the four self-report items: “who/what came to mind” when you responded to this item? Thematic analysis of this cognitive debriefing data suggested racial/ethnic and sexual identity differences in association with these items, including that white women described being visually evaluated as “normal looking” in public settings, while women of color described facing stereotyping, dehumanization, and discrimination. These data suggest that the items’ original design limited visual evaluations in public settings to those regarding attractiveness, indicating that experiences of social and political inequality have been excluded from the scope of research on body esteem. Together, the findings from these studies suggest that when researchers develop self-report items, greater attention is needed to ensure that design choices related to conceptual focus, wording, framing, and imagery do not contain biased information that may be disseminated to the public.