Nonresponse and Measurement Error in Mobile Phone Surveys.

Abstract

Landline telephone surveys have been used for several decades to generate critical knowledge about consumer confidence, health conditions, political attitudes, and other characteristics of the American public. The coverage provided by this methodology is rapidly declining due to widespread adoption and, in many cases, substitution of mobile (cell) phones over landlines. In order to address this problem, survey researchers have begun supplementing landline surveys with samples of cell phone numbers. The error properties of cell phone surveys, particularly with respect to nonresponse and measurement, are largely unknown. Researchers have limited knowledge as to why some people answer surveys on their cell phone but others do not. It is also an open question as to whether people respond less accurately on a cell phone as compared to a landline. The potential to interview people outside the home or engaged in an activity that distracts from the task of responding could result in respondents taking more cognitive shortcuts and providing less accurate data relative to landline interviews. These dynamics could also reduce the reliability of survey estimates and, for some measures, even change the mean of the response distribution. This dissertation uses data from a unique repeated-measures experiment to address these research gaps. Nonresponse modeling indicates that the sets of factors influencing participation decisions in landline and cell phone surveys are different, though overlapping. Measurement error comparisons show that the quality of data from cell phone and landline interviews are generally comparable, with some intriguing exceptions. Finally, there is evidence that respondents may answer some survey questions differently depending on whether they are interviewed at home or away from home, presumably because of differential environmental cues. This research demonstrates that the error properties of landline and cell phone survey data tend to be similar, but there are potentially important exceptions that warrant methodologists’ attention.