Challenges to validity in single‐group interrupted time series analysis

Abstract

Rationale, aims and objectivesSingle‐group interrupted time series analysis (ITSA) is a popular evaluation methodology in which a single unit of observation is studied; the outcome variable is serially ordered as a time series, and the intervention is expected to “interrupt” the level and/or trend of the time series, subsequent to its introduction. The most common threat to validity is history—the possibility that some other event caused the observed effect in the time series. Although history limits the ability to draw causal inferences from single ITSA models, it can be controlled for by using a comparable control group to serve as the counterfactual.MethodTime series data from 2 natural experiments (effect of Florida’s 2000 repeal of its motorcycle helmet law on motorcycle fatalities and California’s 1988 Proposition 99 to reduce cigarette sales) are used to illustrate how history biases results of single‐group ITSA results—as opposed to when that group’s results are contrasted to those of a comparable control group.ResultsIn the first example, an external event occurring at the same time as the helmet repeal appeared to be the cause of a rise in motorcycle deaths, but was only revealed when Florida was contrasted with comparable control states. Conversely, in the second example, a decreasing trend in cigarette sales prior to the intervention raised question about a treatment effect attributed to Proposition 99, but was reinforced when California was contrasted with comparable control states.ConclusionsResults of single‐group ITSA should be considered preliminary, and interpreted with caution, until a more robust study design can be implemented.