Graphical displays for assessing covariate balance in matching studies
dc.contributor.author | Linden, Ariel | en_US |
dc.date.accessioned | 2015-04-02T15:12:33Z | |
dc.date.available | 2016-05-10T20:26:28Z | en |
dc.date.issued | 2015-04 | en_US |
dc.identifier.citation | Linden, Ariel (2015). "Graphical displays for assessing covariate balance in matching studies." Journal of Evaluation in Clinical Practice (2): 242-247. | en_US |
dc.identifier.issn | 1356-1294 | en_US |
dc.identifier.issn | 1365-2753 | en_US |
dc.identifier.uri | https://hdl.handle.net/2027.42/110864 | |
dc.description.abstract | Rationale, aims and objectivesAn essential requirement for ensuring the validity of outcomes in matching studies is that study groups are comparable on observed pre‐intervention characteristics. Investigators typically use numerical diagnostics, such as t‐tests, to assess comparability (referred to as ‘balance’). However, such diagnostics only test equality along one dimension (e.g. means in the case of t‐tests), and therefore do not adequately capture imbalances that may exist elsewhere in the distribution. Furthermore, these tests are generally sensitive to sample size, raising the concern that a reduction in power may be mistaken for an improvement in covariate balance. In this paper, we demonstrate the shortcomings of numerical diagnostics and demonstrate how visual displays provide a complete representation of the data to more robustly assess balance.MethodsWe generate artificial datasets specifically designed to demonstrate how widely used equality tests capture only a single‐dimension of the data and are sensitive to sample size. We then plot the covariate distributions using several graphical displays.ResultsAs expected, tests showing perfect covariate balance in means failed to reflect imbalances at higher moments (variances). However, these discrepancies were easily detected upon inspection of the graphic displays. Additionally, smaller sample sizes led to the appearance of covariate balance, when in fact it was a result of lower statistical power.ConclusionsGiven the limitations of numerical diagnostics, we advocate using graphical displays for assessing covariate balance and encourage investigators to provide such graphs when reporting balance statistics in their matching studies. | en_US |
dc.publisher | Wiley Periodicals, Inc. | en_US |
dc.publisher | Oxford University Press | en_US |
dc.subject.other | covariate balance | en_US |
dc.subject.other | matching | en_US |
dc.subject.other | propensity score | en_US |
dc.subject.other | standardized differences | en_US |
dc.subject.other | visual displays | en_US |
dc.title | Graphical displays for assessing covariate balance in matching studies | en_US |
dc.type | Article | en_US |
dc.rights.robots | IndexNoFollow | en_US |
dc.subject.hlbsecondlevel | Medicine (General) | en_US |
dc.subject.hlbtoplevel | Health Sciences | en_US |
dc.description.peerreviewed | Peer Reviewed | en_US |
dc.description.bitstreamurl | http://deepblue.lib.umich.edu/bitstream/2027.42/110864/1/jep12297.pdf | |
dc.identifier.doi | 10.1111/jep.12297 | en_US |
dc.identifier.source | Journal of Evaluation in Clinical Practice | en_US |
dc.identifier.citedreference | Tukey, J. W. ( 1977 ) Exploratory Data Analysis. Reading, MA: Addison–Wesley. | en_US |
dc.identifier.citedreference | Rubin, D. B. ( 1973 ) Matching to remove bias in observational studies. Biometrics, 29, 159 – 184. | en_US |
dc.identifier.citedreference | Austin, P. C. ( 2009 ) Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity‐score matched samples. Statistics in Medicine, 28, 3083 – 3107. | en_US |
dc.identifier.citedreference | Stuart, E. A. ( 2010 ) Matching methods for causal inference: a review and a look forward. Statistical Science, 25 ( 1 ), 1 – 21. | en_US |
dc.identifier.citedreference | Linden, A. & Samuels, S. J. ( 2013 ) Using balance statistics to determine the optimal number of controls in matching studies. Journal of Evaluation in Clinical Practice, 19 ( 5 ), 968 – 975. | en_US |
dc.identifier.citedreference | Linden, A. ( 2008 ) Sample size in disease management program evaluation: the challenge of demonstrating a statistically significant reduction in admissions. Disease Management, 11 ( 2 ), 95 – 101. | en_US |
dc.identifier.citedreference | Flury, B. K. & Reidwyl, H. ( 1986 ) Standard distance in univariate and multivariate analysis. The American Statistician, 40, 249 – 251. | en_US |
dc.identifier.citedreference | Kolmogorov, A. N. ( 1933 ) Sulla determinazione empirica di una legge di distribuzione. Giornale dell’ Istituto Italiano degli Attuari, 4, 83 – 91. | en_US |
dc.identifier.citedreference | Smirnov, N. V. ( 1933 ) Estimate of deviation between empirical distribution functions in two independent samples. Bulletin Moscow University, 2, 3 – 16. | en_US |
dc.identifier.citedreference | Anderson, T. W. & Darling, D. A. ( 1952 ) Asymptotic theory of certain ‘goodness of fit’ criteria based on stochastic processes. The Annals of Mathematical Statistics, 23 ( 2 ), 193 – 212. | en_US |
dc.identifier.citedreference | Wilcoxon, F. ( 1945 ) Individual comparisons by ranking methods. Biometrics, 1, 80 – 83. | en_US |
dc.identifier.citedreference | Bland, M. ( 2000 ) An Introduction to Medical Statistics, 3rd edn. Oxford: Oxford University Press. | en_US |
dc.identifier.citedreference | Levene, H. ( 1960 ) Robust tests for equality of variances. In Contributions to Probability and Statistics: Essays in Honor of Harold Hotelling (eds I. Olkin, S. G. Ghurye, W. Hoeffding, W. G. Madow & H. B. Mann ), pp. 278 – 292. Menlo Park, CA: Stanford University Press. | en_US |
dc.identifier.citedreference | Chambers, J. M., Cleveland, W. S., Kleiner, B. & Tukey, P. A. ( 1983 ) Graphical Methods for Data Analysis. Belmont, CA: Wadsworth. | en_US |
dc.identifier.citedreference | Wilk, M. B. & Gnanadesikan, R. ( 1968 ) Probability plotting methods for the analysis of data. Biometrika, 55, 1 – 17. | en_US |
dc.identifier.citedreference | StataCorp ( 2013 ) Stata 13 Base Reference Manual. College Station, TX: Stata Press. | en_US |
dc.identifier.citedreference | Linden, A. ( 2014 ) qqplot3: Stata module for plotting unweighted and weighted Q‐Q plots. Available at: http://ideas.repec.org/c/boc/bocode/s457856.html (last accessed 1 November 2014). | en_US |
dc.identifier.citedreference | Cox, N. J. ( 1999 ) gr42: quantile plots, generalized. Stata Technical Bulletin, 51, 16 – 18. | en_US |
dc.identifier.citedreference | Nichols, A. ( 2010 ) byhist: Stata module to graph interlaced histograms: for comparing histograms by a categorical variable. Available at: http://ideas.repec.org/c/boc/bocode/s456982.html (last accessed 1 November 2014). | en_US |
dc.identifier.citedreference | Rosenbaum, P. R. & Rubin, D. B. ( 1983 ) The central role of propensity score in observational studies for causal effects. Biometrika, 70, 41 – 55. | en_US |
dc.identifier.citedreference | McCaffrey, D., Ridgeway, G. & Morral, A. ( 2004 ) Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychological Methods, 9 ( 4 ), 403 – 425. | en_US |
dc.owningcollname | Interdisciplinary and Peer-Reviewed |
Files in this item
Remediation of Harmful Language
The University of Michigan Library aims to describe library materials in a way that respects the people and communities who create, use, and are represented in our collections. Report harmful or offensive language in catalog records, finding aids, or elsewhere in our collections anonymously through our metadata feedback form. More information at Remediation of Harmful Language.
Accessibility
If you are unable to use this file in its current format, please select the Contact Us link and we can modify it to make it more accessible to you.