Satisficing in Web Surveys: Implications for Data Quality and Strategies for Reduction.

Abstract

With the increasing use of the Web in mixed mode surveys, especially those conducted by the Census and other federal statistical agencies, it has become more urgent than ever to develop methods to enhance online measurement quality. This dissertation research (including three studies) focuses on respondent satisficing as a source of online measurement errors, and interactive intervention to reduce satisficing behaviors. The first study evaluates speeding (or very fast responding) as an indicator by investigating how it is associated with another well-known satisficing behavior – non-differentiation in grid questions. The second and third studies examine intervention design in Web surveys to curtail respondent satisficing. Specifically, the second study examines whether intervention for different satisficing behaviors could produce different effects on overall response quality. The third study explores whether intervention in Web surveys can induce the feeling of interacting with a human agent. Study 1 shows that respondents who speed more often tend to straightline on more grid questions, suggesting that the tendency to speed is indeed related to satisficing. The results of Study 2 demonstrate that intervention in a survey can have a broad impact of improving respondents’ reporting effort, which is not restricted to the satisficing behavior it targets nor the type of survey questions where it occurs. The different intervention designs in Study 3 did not yield consistent differences in respondent behaviors. However, the intervention conditions, regardless of the design, produced more reports of socially desirable answers compared to the no-intervention condition. This pair of observations – that intervention can help increase respondent effort (Study 2) but also make respondents less willing to disclose undesirable information (Study 3) – seem to converge on one explanation on how intervention works. That is, the interactive feedback about respondents’ behaviors may increase their sense of social presence as they complete the online questionnaire. As a result, this may motivate respondents to present themselves in a more positive light as a respondent (by working harder on the survey) as well as a person (by not reporting undesirable information about themselves).