Fast times and easy questions: the effects of age, experience and question complexity on web survey response times

Abstract

This paper examines response times (RT) to survey questions. Cognitive psychologists have long relied on response times to study cognitive processes but response time data have only recently received attention from survey researchers. To date, most of the studies on response times in surveys have treated response times either as a predictor or as a proxy measure for some other variable (e.g. attitude accessibility) of greater interest. As a result, response times have not been the main focus of the research. Focusing on the nature and determinants of response times, this paper examines variables that affect how long it takes respondents to answer questions in web surveys. Using the survey response model proposed by Tourangeau, Rips, and Rasinski (2000), we include both item-level characteristics and respondent-level characteristics thought to affect response times in a two-level cross-classified model. Much of the time spent on processing the questions involves reading and interpreting them. The results from the cross-classified models indicate that response times are affected by question characteristics such as the total number of clauses and the number of words per clause that probably reflect reading times. In addition, response times are also affected by the number and type of answer categories, and the location of the question within the questionnaire, as well as respondent characteristics such as age, education and experience with the Internet and with completing web surveys. Aside from their fixed effects on response times, respondent-level characteristics (such as age) are shown to vary randomly over questions and effects of question-level characteristics (such as types of questions and response scales) vary randomly over respondents. Copyright © 2007 John Wiley & Sons, Ltd.