Comparison of static and rolling logistic regression models on predicting invasive mechanical ventilation or death from COVID-19—A retrospective, multicentre study
dc.contributor.author | Engoren, Milo | |
dc.contributor.author | Pancaro, Carlo | |
dc.contributor.author | Yeldo, Nicholas S. | |
dc.contributor.author | Kerzabi, Lotfi S. | |
dc.contributor.author | Douville, Nicholas | |
dc.date.accessioned | 2023-02-01T18:57:58Z | |
dc.date.available | 2024-02-01 13:57:57 | en |
dc.date.available | 2023-02-01T18:57:58Z | |
dc.date.issued | 2023-01 | |
dc.identifier.citation | Engoren, Milo; Pancaro, Carlo; Yeldo, Nicholas S.; Kerzabi, Lotfi S.; Douville, Nicholas (2023). "Comparison of static and rolling logistic regression models on predicting invasive mechanical ventilation or death from COVID-19—A retrospective, multicentre study." The Clinical Respiratory Journal 17(1): 40-49. | |
dc.identifier.issn | 1752-6981 | |
dc.identifier.issn | 1752-699X | |
dc.identifier.uri | https://hdl.handle.net/2027.42/175759 | |
dc.description.abstract | IntroductionCOVID-19 virus has undergone mutations, and the introduction of vaccines and effective treatments have changed its clinical severity. We hypothesized that models that evolve may better predict invasive mechanical ventilation or death than do static models.MethodsThis retrospective study of adult patients with COVID-19 from six Michigan hospitals analysed 20 demographic, comorbid, vital sign and laboratory factors, one derived factor and nine factors representing changes in vital signs or laboratory values with time for their ability to predict death or invasive mechanical ventilation within the next 4, 8 or 24 h. Static logistic regression was constructed on the initial 300 patients and tested on the remaining 6741 patients. Rolling logistic regression was similarly constructed on the initial 300 patients, but then new patients were added, and older patients removed. Each new construction model was subsequently tested on the next patient. Static and rolling models were compared with receiver operator characteristic and precision-recall curves.ResultsOf the 7041 patients, 534 (7.6%) required invasive mechanical ventilation or died within 14 days of arrival. Rolling models improved discrimination (0.865 ± 0.010, 0.856 ± 0.007 and 0.843 ± 0.005 for the 4, 8 and 24-h models, respectively; all p < 0.001 compared with the static logistic regressions with 0.827 ± 0.011, 0.794 ± 0.012 and 0.735 ± 0.012, respectively). Similarly, the areas under the precision-recall curves improved from 0.006, 0.010 and 0.021 with the static models to 0.030, 0.045 and 0.076 for the 4-, 8- and 24-h rolling models, respectively, all p < 0.001.ConclusionRolling models with contemporaneous data maintained better metrics of performance than static models, which used older data.When a disease or treatment is rapidly changing, traditional (static) predictive models may lose their accuracy. We show that in 7041 patients of whom 534 (7.6%) required invasive mechanical ventilation or died within 14 days that rolling logistic regression models improved both discrimination and the areas under the precision-recall curves—all p < 0.001. | |
dc.publisher | Springer | |
dc.publisher | Wiley Periodicals, Inc. | |
dc.subject.other | mechanical ventilation | |
dc.subject.other | logistic regression | |
dc.subject.other | death, clinical decision support, clinical prediction models | |
dc.subject.other | COVID | |
dc.title | Comparison of static and rolling logistic regression models on predicting invasive mechanical ventilation or death from COVID-19—A retrospective, multicentre study | |
dc.type | Article | |
dc.rights.robots | IndexNoFollow | |
dc.subject.hlbsecondlevel | Pulmonary Medicine | |
dc.subject.hlbtoplevel | Health Sciences | |
dc.description.peerreviewed | Peer Reviewed | |
dc.description.bitstreamurl | http://deepblue.lib.umich.edu/bitstream/2027.42/175759/1/crj13560_am.pdf | |
dc.description.bitstreamurl | http://deepblue.lib.umich.edu/bitstream/2027.42/175759/2/crj13560.pdf | |
dc.identifier.doi | 10.1111/crj.13560 | |
dc.identifier.source | The Clinical Respiratory Journal | |
dc.identifier.citedreference | Williams B (ed) National Early Warning Score (NEWS) 2 – standardising the assessment of acute illness severity in the NHS. ISBN 978-1-86016-682-2. 2017. | |
dc.identifier.citedreference | Dennis JM, McGovern AP, Vollmer SJ, Mateen BA. Improving survival of critical care patients with coronavirus disease 2019 in England: a national cohort study, March to June 2020. Crit Care Med. 2021; 49 ( 2 ): 209 - 214. doi: 10.1097/CCM.0000000000004747 | |
dc.identifier.citedreference | Hasan MN, Haider N, Stigler FL, et al. The global case-fatality rate of covid-19 has been declining since May 2020. Am J Trop Med Hyg. 2021; 104 ( 6 ): 2176 - 2184. doi: 10.4269/ajtmh.20-1496 | |
dc.identifier.citedreference | Yeates EO, Nahmias J, Chinn J, et al. Improved outcomes over time for adult COVID-19 patients with acute respiratory distress syndrome or acute respiratory failure. PLoS One. 2021; 16 ( 6 ): e0253767. doi: 10.1371/journal.pone.0253767 | |
dc.identifier.citedreference | Kulkarni AR, Athavale AM, Sahni A, et al. Deep learning model to predict the need for mechanical ventilation using chest X-ray images in hospitalised patients with COVID-19. BMJ Innov. 2021; 7 ( 2 ): 261 - 270. doi: 10.1136/bmjinnov-2020-000593 | |
dc.identifier.citedreference | Douville NJ, Douville CB, Mentz G, et al. Clinically applicable approach for predicting mechanical ventilation in patients with COVID-19. Br J Anaesth. 2021; 126 ( 3 ): 578 - 589. doi: 10.1016/j.bja.2020.11.034 | |
dc.identifier.citedreference | O’Reilly Nugent A, Kelly PT, Stanton J, Swanney MP, Graham B, Beckert L. Measuring tracheal oxygen concentration delivered via nasal cannulae by tracheal sampling. Respirology. 2014; 19 ( 4 ): 538 - 543. doi: 10.1111/resp.12268 | |
dc.identifier.citedreference | Saito T, Rehmsmeier M. The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS One. 2015; 10 ( 3 ): e0118432. doi: 10.1371/journal.pone.0118432 | |
dc.identifier.citedreference | Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982; 143 ( 1 ): 29 - 36. doi: 10.1148/radiology.143.1.7063747 | |
dc.identifier.citedreference | Boyd K, Eng KH, Page CD. Area under the precision-recall curve: point estimates and confidence intervals. In: Blockeel H, Kersting K, Nijssen S, Železný F, eds. Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2013. Lecture Notes in Computer Science. Vol. 8190. Berlin, Heidelberg: Springer. | |
dc.identifier.citedreference | Kitzis SN, Kelley H, Berg E, Massaro DW, Friedman D. Broadening the tests of learning models. J Math Psychol. 1998; 42 ( 2-3 ): 327 - 355. doi: 10.1006/jmps.1998.1215 | |
dc.identifier.citedreference | Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996; 49 ( 12 ): 1373 - 1379. doi: 10.1016/S0895-4356(96)00236-3 | |
dc.identifier.citedreference | Dennis JM, McGovern AP, Thomas NJ, Wilde H, Vollmer SJ, Mateen BA. Trends in 28-day mortality of critical care patients with Coronavirus Disease 2019 in the United Kingdom: a national cohort study, March 2020 to January 2021. Crit Care Med. 2021; 49 ( 11 ): 1895 - 1900. doi: 10.1097/CCM.0000000000005184 | |
dc.identifier.citedreference | Raad M, Gorgis S, Abshire C, et al. COVID-19 risk index (CRI): a simple and validated emergency department risk score that predicts mortality and the need for mechanical ventilation. J Thromb Thrombolysis. 2022; 53 ( 3 ): 567 - 575. doi: 10.1007/s11239-021-02565-6 | |
dc.identifier.citedreference | Alberdi-Iglesias A, Martín-Rodríguez F, Ortega Rabbione G, et al. Role of SpO2/FiO2 ratio and ROX index in predicting early invasive mechanical ventilation in COVID-19. A pragmatic, retrospective, multi-center study. Biomedicines. 2021; 9: 1036. | |
dc.identifier.citedreference | Shimizu M, Hashimoto S. Peripheral oxygen saturation to inspiratory oxygen fraction ratio-based identification of critically ill coronavirus disease patients for early therapeutic interventions. J Anesth. 2021; 35 ( 6 ): 827 - 836. doi: 10.1007/s00540-021-02986-w | |
dc.identifier.citedreference | American College of Chest Physicians/Society of Critical Care Medicine Consensus Conference: definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. Crit Care Med. 1992; 20 ( 6 ): 864 - 874. doi: 10.1097/00003246-199206000-00025 | |
dc.identifier.citedreference | Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified early warning score in medical admissions. QJM. 2001; 94 ( 10 ): 521 - 526. doi: 10.1093/qjmed/94.10.521 | |
dc.identifier.citedreference | Angus DC, Seymour CW, Coopersmith CM, et al. A framework for the development and interpretation of different sepsis definitions and clinical criteria. Crit Care Med. 2016; 44 ( 3 ): e113 - e121. doi: 10.1097/CCM.0000000000001730 | |
dc.identifier.citedreference | Ho FK, Petermann-Rocha F, Gray SR, et al. Is older age associated with COVID-19 mortality in the absence of other risk factors? General population cohort study of 470,034 participants. PLoS One. 2020; 15 ( 11 ): e0241824. doi: 10.1371/journal.pone.0241824 | |
dc.identifier.citedreference | Nguyen NT, Chinn J, De Ferrante M, Kirby KA, Hohmann SF, Amin A. Male gender is a predictor of higher mortality in hospitalized adults with COVID-19. PLoS One. 2021; 16 ( 7 ): e0254066. doi: 10.1371/journal.pone.0254066 | |
dc.identifier.citedreference | https://www.kff.org/racial-equity-and-health-policy/issue-brief/covid-19-cases-and-deaths-by-race-ethnicity-current-data-and-changes-over-time/ accessed Feb 3, 2022. | |
dc.identifier.citedreference | https://www.cdc.gov/nhsn/covid19/ltc-report-overview.html accessed Feb 3, 2022. | |
dc.identifier.citedreference | Knaus WA, Wagner DP, Draper EA, et al. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest. 1991; 100 ( 6 ): 1619 - 1636. doi: 10.1378/chest.100.6.1619 | |
dc.identifier.citedreference | WHO Rapid Evidence Appraisal for COVID-19 Therapies Working Group, Sterne JAC, Murthy S, et al. Association between administration of systemic corticosteroids and mortality among critically ill patients with COVID-19: a meta-analysis. JAMA. 2020; 324: 1330 - 1341. | |
dc.identifier.citedreference | Jiang S, Hillyer C, Du L. Neutralizing antibodies against SARS-CoV-2 and other human coronaviruses. Trends Immunol. 2020; 41 ( 5 ): 355 - 359. doi: 10.1016/j.it.2020.03.007 | |
dc.identifier.citedreference | Baden LR, El Sahly HM, Essink B, et al. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N Engl J Med. 2021; 384 ( 5 ): 403 - 416. doi: 10.1056/NEJMoa2035389 | |
dc.identifier.citedreference | Radovanovic D, Coppola S, Franceschi E, et al. Mortality and clinical outcomes in patients with COVID-19 pneumonia treated with non-invasive respiratory support: a rapid review. J Crit Care. 2021; 65: 1 - 8. doi: 10.1016/j.jcrc.2021.05.007 | |
dc.identifier.citedreference | Ren SY, Wang WB, Gao RD, Zhou A-M. Omicron variant (B.1.1.529) of SARS-CoV-2: mutation, infectivity, transmission, and vaccine resistance. World J Clin Cases. 2022; 10 ( 1 ): 1 - 11. doi: 10.12998/wjcc.v10.i1.1 | |
dc.identifier.citedreference | Carbonell R, Urgelés S, Rodríguez A, et al. Mortality comparison between the first and second/third waves among 3,795 critical COVID-19 patients with pneumonia admitted to the ICU: a multicentre retrospective cohort study. Lancet Reg Health Eur. 2021; 11: 100243. doi: 10.1016/j.lanepe.2021.100243 | |
dc.working.doi | NO | en |
dc.owningcollname | Interdisciplinary and Peer-Reviewed |
Files in this item
Remediation of Harmful Language
The University of Michigan Library aims to describe library materials in a way that respects the people and communities who create, use, and are represented in our collections. Report harmful or offensive language in catalog records, finding aids, or elsewhere in our collections anonymously through our metadata feedback form. More information at Remediation of Harmful Language.
Accessibility
If you are unable to use this file in its current format, please select the Contact Us link and we can modify it to make it more accessible to you.