Standardized Hospitalization Ratio: Modeling, Sequential Control of False Discovery Rates, and Continuous Monitoring

Abstract

Healthcare provider profiling is of nationwide importance. For providers with comparably poor outcomes, surveillance and financial penalties may ensue. Moreover, because COVID-19 has dramatically changed how healthcare is delivered, profiling methods must account for the temporal variation caused by the pandemic to accurately evaluate provider performance. However, analyzing periodically updated national dialysis databases with extremely large sample sizes, high-dimensional parameters, and dynamic heterogeneity creates modeling challenges. In addition, dynamic monitoring of providers over time while sequentially controlling false discovery rates (FDR) further complicates the problems. Finally, most statistical methods used in provider profiling treat all risk-adjusted variability between providers as due to quality of care, which is often untrue. The resulting algorithms flag too many providers and unfairly penalize large providers.

In this dissertation, with hospital admission as an example, we propose new methods to model the provider effects, to address the issues in a longitudinal testing process while adjusting for unexplained between-provider variation due to unmeasured risk factors, and to continuously monitor providers.

In Chapter II, to account for temporal variation in provider profiling and accurately evaluate providers’ performance during the pandemic, we propose a computationally efficient recurrent event modeling procedure to capture various trends of hospitalizations in multiple important time scales. The proposed method is assessed under a range of simulations and demonstrates substantially improved computational efficiency over existing R packages. Our work also addresses the main disadvantages of existing indirect and direct standardization methods for provider profiling.

In Chapter III, to sequentially monitor and profile providers over time, we propose using the Shewhart chart based on z-scores. This sequential monitoring process, however, does create three main challenges. First, the common assumption in provider profiling that the entire risk-adjusted variability between providers is due to quality of care is often violated; consequently, the algorithms unfairly penalize large providers. Second, in this repeated test framework, a major problem is the high false discovery rate (FDR). Finally, national dialysis databases are updated frequently, with the facility effect and baseline rate varying over time. In response, we modify the Shewhart chart as follows: we adopt individualized empirical null methods to account for unmeasured confounders; we utilize online testing rules to determine control limits that reduce the rate of false discoveries over time; and we propose online updating methods to estimate the model parameters to allow for changing facility effects and baseline rate. We simulate recurrent event data that mimic our motivating real data, and evaluate our method with various sets of online testing rules; our method demonstrates substantially improved FDR, which is more uniform across different sample sizes. Finally, we show an important application to monitoring U.S. dialysis facilities over time.

In Chapter IV, to enable early detection of unacceptable performance, and continuously monitor each facility’s hospital admissions, we propose Observed–Expected (O–E) Cumulative SUM (CUSUM) control charts using a likelihood ratio test based on the pseudo likelihood of a semiparametric proportional rate model. In addition, in applications of recurrent events, due to unmeasured confounders and correlation within patients, the variance is typically greater than the nominal level, leading to a great number of flags, many of which are false discoveries, especially for large providers. We utilize the empirical null to address this issue. The visual monitoring provided by the CUSUM chart helps provide timely feedback to providers.