The impact of survey mode on US national estimates of adolescent drug prevalence: results from a randomized controlled study

Abstract

Background and AimsIncreasing numbers of school‐based drug surveys are transitioning data collection to electronic tablets from paper‐and‐pencil, which may produce a survey mode effect and consequent discontinuity in time trends for population estimates of drug prevalence. This study tested whether (a) overall, self‐reported drug use prevalence is higher on electronic tablets versus paper‐and‐pencil surveys, (b) socio‐demographics moderate survey mode effects and (c) levels of missing data are lower for electronic tablet versus paper‐and‐pencil modes.DesignA randomized controlled experiment.SettingResults are nationally representative of students in the contiguous United States.ParticipantsA total of 41 866 8th, 10th and 12th grade students who participated in the 2019 Monitoring the Future school‐based survey administration.Intervention and comparatorSurveys were administered to students in a randomly selected half of schools with electronic tablets (intervention) and with paper‐and‐pencil format (comparator) for the other half.MeasurementsPrimary outcome was the total number of positive drug use responses. Secondary outcomes were the percentage of respondents completing all drug questions, percentage of drug questions unanswered and mean number of missing drug items.FindingsThe relative risk (RR) for total number of positive drug use responses for electronic tablets versus paper‐and‐pencil surveys were small and their 95% confidence intervals (CI) included the value of one for reporting intervals of life‐time (RR = 1.03; 95% CI, 0.93–1.14), past 12 months (RR = 1.01; 95% CI, 0.91–1.11), past 30 days (RR = 1.05; 95% CI, 0.93–1.20) and for heavy use (RR = 1.10; 95% CI, 0.93–1.29). Multiplicative interaction tests indicated no moderation of these relative risks by race (white versus non‐white), population density, census region, public/private school, year of school participation, survey version or non‐complete drug responses. Levels of missing data were significantly lower for electronic tablets versus paper‐and‐pencil surveys.ConclusionsAdolescent drug prevalence estimates in the United States differed little across electronic tablet versus paper‐and‐pencil survey modes, and showed little to no effect modification by socio‐demographics. Levels of missing data were lower for electronic tablets.