Comparison of static and rolling logistic regression models on predicting invasive mechanical ventilation or death from COVID-19—A retrospective, multicentre study

Abstract

IntroductionCOVID-19 virus has undergone mutations, and the introduction of vaccines and effective treatments have changed its clinical severity. We hypothesized that models that evolve may better predict invasive mechanical ventilation or death than do static models.MethodsThis retrospective study of adult patients with COVID-19 from six Michigan hospitals analysed 20 demographic, comorbid, vital sign and laboratory factors, one derived factor and nine factors representing changes in vital signs or laboratory values with time for their ability to predict death or invasive mechanical ventilation within the next 4, 8 or 24 h. Static logistic regression was constructed on the initial 300 patients and tested on the remaining 6741 patients. Rolling logistic regression was similarly constructed on the initial 300 patients, but then new patients were added, and older patients removed. Each new construction model was subsequently tested on the next patient. Static and rolling models were compared with receiver operator characteristic and precision-recall curves.ResultsOf the 7041 patients, 534 (7.6%) required invasive mechanical ventilation or died within 14 days of arrival. Rolling models improved discrimination (0.865 ± 0.010, 0.856 ± 0.007 and 0.843 ± 0.005 for the 4, 8 and 24-h models, respectively; all p < 0.001 compared with the static logistic regressions with 0.827 ± 0.011, 0.794 ± 0.012 and 0.735 ± 0.012, respectively). Similarly, the areas under the precision-recall curves improved from 0.006, 0.010 and 0.021 with the static models to 0.030, 0.045 and 0.076 for the 4-, 8- and 24-h rolling models, respectively, all p < 0.001.ConclusionRolling models with contemporaneous data maintained better metrics of performance than static models, which used older data.When a disease or treatment is rapidly changing, traditional (static) predictive models may lose their accuracy. We show that in 7041 patients of whom 534 (7.6%) required invasive mechanical ventilation or died within 14 days that rolling logistic regression models improved both discrimination and the areas under the precision-recall curves—all p < 0.001.