Subgroup analysis and adaptive experiments crave for debiasing

Abstract

Results obtained from reliably designed randomized experiments are often considered to be evidence of the highest grade for assessing the effectiveness of biomedical or behavioral interventions. Nevertheless, even with randomized experiments, statistical bias can arise in post hoc analysis of the data or through adaptive data collection. In this article, we discuss the need for and review some of the recent developments in statistical methodologies to address the issue of potential bias in adaptive experiments and in subgroup analysis. For adaptive experiments, we focus on adaptive treatment assignments. For subgroup analysis, we focus on post hoc subgroup selection and review several frequentist approaches for debiased inference on the best-selected subgroup effects.This article is categorized under:Applications of Computational Statistics > Clinical TrialsIn this review, we review three randomized experimental design strategies, including completely randomized experiments, covariate adaptive experiments, and response adaptive experiments. Furthermore, we categorize the subgroup analysis literature into three types: exploratory, debiased, and confirmatory subgroup analysis. The later part of this review focuses on discussing the methodologies used for conducting debiased subgroup analysis.