Machine learning and modeling: Data, validation, communication challenges
dc.contributor.author | El Naqa, Issam | |
dc.contributor.author | Ruan, Dan | |
dc.contributor.author | Valdes, Gilmer | |
dc.contributor.author | Dekker, Andre | |
dc.contributor.author | McNutt, Todd | |
dc.contributor.author | Ge, Yaorong | |
dc.contributor.author | Wu, Q. Jackie | |
dc.contributor.author | Oh, Jung Hun | |
dc.contributor.author | Thor, Maria | |
dc.contributor.author | Smith, Wade | |
dc.contributor.author | Rao, Arvind | |
dc.contributor.author | Fuller, Clifton | |
dc.contributor.author | Xiao, Ying | |
dc.contributor.author | Manion, Frank | |
dc.contributor.author | Schipper, Matthew | |
dc.contributor.author | Mayo, Charles | |
dc.contributor.author | Moran, Jean M. | |
dc.contributor.author | Ten Haken, Randall | |
dc.date.accessioned | 2018-11-20T15:32:34Z | |
dc.date.available | 2019-12-02T14:55:09Z | en |
dc.date.issued | 2018-10 | |
dc.identifier.citation | El Naqa, Issam; Ruan, Dan; Valdes, Gilmer; Dekker, Andre; McNutt, Todd; Ge, Yaorong; Wu, Q. Jackie; Oh, Jung Hun; Thor, Maria; Smith, Wade; Rao, Arvind; Fuller, Clifton; Xiao, Ying; Manion, Frank; Schipper, Matthew; Mayo, Charles; Moran, Jean M.; Ten Haken, Randall (2018). "Machine learning and modeling: Data, validation, communication challenges." Medical Physics 45(10): e834-e840. | |
dc.identifier.issn | 0094-2405 | |
dc.identifier.issn | 2473-4209 | |
dc.identifier.uri | https://hdl.handle.net/2027.42/146326 | |
dc.publisher | McGraw‐Hill | |
dc.publisher | Wiley Periodicals, Inc. | |
dc.subject.other | radiation oncology | |
dc.subject.other | machine learning | |
dc.subject.other | big data | |
dc.title | Machine learning and modeling: Data, validation, communication challenges | |
dc.type | Article | en_US |
dc.rights.robots | IndexNoFollow | |
dc.subject.hlbsecondlevel | Medicine (General) | |
dc.subject.hlbtoplevel | Health Sciences | |
dc.description.peerreviewed | Peer Reviewed | |
dc.description.bitstreamurl | https://deepblue.lib.umich.edu/bitstream/2027.42/146326/1/mp12811_am.pdf | |
dc.description.bitstreamurl | https://deepblue.lib.umich.edu/bitstream/2027.42/146326/2/mp12811.pdf | |
dc.identifier.doi | 10.1002/mp.12811 | |
dc.identifier.source | Medical Physics | |
dc.identifier.citedreference | Jiang B, Zhang X, Cai T. Estimating the confidence interval for prediction errors of support vector machine classifiers. J Mach Learn Res. 2008; 9: 521 – 540. | |
dc.identifier.citedreference | Györfi LS, Ottucsák GG, Walk H. Machine Learning for Financial Engineering. Singapore; London: World Scientific; 2012. | |
dc.identifier.citedreference | Sheng Y, Ge Y, Yuan L, Li T, Yin FF, Wu QJ. Outlier identification in radiation therapy knowledge‐based planning: a study of pelvic cases. Med Phys. 2017; 44: 5617 – 5626. | |
dc.identifier.citedreference | El Naqa I, Bradley JD, Lindsay PE, Hope AJ, Deasy JO. Predicting radiotherapy outcomes using statistical learning techniques. Phys Med Biol. 2009; 54: S9 – S30. | |
dc.identifier.citedreference | Britannica TEOE. Occam’s razor. In Encyclopædia Britannica,. London, UK: Encyclopædia Britannica, inc.; 2015. | |
dc.identifier.citedreference | Goodfellow I, Bengio Y, Courville A. Deep Learning. Cambridge, MA: MIT Press; 2017. | |
dc.identifier.citedreference | Cherkassky VS, Mulier F. Learning From Data: Concepts, Theory, and Methods. 2nd ed. Hoboken, NJ: IEEE Press: Wiley‐Interscience; 2007. | |
dc.identifier.citedreference | Lian J, Yuan L, Ge Y, et al. Modeling the dosimetry of organ‐at‐risk in head and neck IMRT planning: an intertechnique and interinstitutional study. Med Phys. 2013; 40: 121704. | |
dc.identifier.citedreference | Gammerman A, Vovk V. Prediction algorithms and confidence measures based on algorithmic randomness theory. Theoret Comput Sci. 2002; 287: 209 – 217. | |
dc.identifier.citedreference | Mitra S. Introduction to Machine Learning and Bioinformatics. Boca Raton: CRC Press; 2008. | |
dc.identifier.citedreference | Bradley AP. The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recogn. 1997; 30: 1145 – 1159. | |
dc.identifier.citedreference | Javanmard A, Montanari A. Confidence intervals and hypothesis testing for high‐dimensional regression. J Mach Learn Res. 2014; 15: 2869 – 2909. | |
dc.identifier.citedreference | Hoeting JA, Madigan D, Raftery AE, Volinsky CT. Bayesian model averaging: a tutorial. Stat Sci. 1999; 14: 382 – 401. | |
dc.identifier.citedreference | Yuan L, Ge Y, Lee WR, Yin FF, Kirkpatrick JP, Wu QJ. Quantitative analysis of the factors which affect the interpatient organ‐at‐risk dose sparing variation in IMRT plans. Med Phys. 2012; 39: 6868 – 6878. | |
dc.identifier.citedreference | Cooper GF, Aliferis CF, Ambrosino R, et al. An evaluation of machine‐learning methods for predicting pneumonia mortality. Artif Intell Med. 1997; 9: 107 – 138. | |
dc.identifier.citedreference | Cooper GF, Abraham V, Aliferis CF, et al. Predicting dire outcomes of patients with community acquired pneumonia. J Biomed Inform. 2005; 38: 347 – 366. | |
dc.identifier.citedreference | Caruana R, Lou Y, Gehrke J, Koch P, Sturm M, Elhadad N. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30‐day readmission. Paper presented at: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 2015. | |
dc.identifier.citedreference | Lou Y, Caruana R, Gehrke J. Intelligible models for classification and regression. Paper presented at: Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining 2012. | |
dc.identifier.citedreference | Valdes G, Luna JM, Eaton E, Simone CB. MediBoost: a patient stratification tool for interpretable decision making in the era of precision medicine. Sci Rep. 2016; 6. | |
dc.identifier.citedreference | Ribeiro MT, Singh S, Guestrin C. Model‐agnostic interpretability of machine learning. arXiv preprint arXiv:160605386. 2016. | |
dc.identifier.citedreference | Luo Y, El Naqa I, McShan DL, et al. Unraveling biophysical interactions of radiation pneumonitis in non‐small‐cell lung cancer via Bayesian network analysis. Radiother Oncol. 2017; 123: 85 – 92. | |
dc.identifier.citedreference | Kang J, Schwartz R, Flickinger J, Beriwal S. Machine learning approaches for predicting radiation therapy outcomes: a clinician’s perspective. Int J Radiat Oncol Biol Phys. 2015; 93: 1127 – 1135. | |
dc.identifier.citedreference | Singh H, Spitzmueller C, Petersen NJ, Sawhney MK, Sittig DF. Information overload and missed test results in electronic health record‐based settings. JAMA Intern Med. 2013; 173: 702 – 704. | |
dc.identifier.citedreference | Duncan J. Information overload: when less is more in medical imaging. De Gruyter. 2017; 4: 179 – 183. | |
dc.identifier.citedreference | Lambin P, van Stiphout RG, Starmans MH, et al. Predicting outcomes in radiation oncology–multifactorial decision support systems. Nat Rev Clin Oncol. 2013; 10: 27 – 40. | |
dc.identifier.citedreference | Smith WP, Kim M, Holdsworth C, Liao J, Phillips MH. Personalized treatment planning with a model of radiation therapy outcomes for use in multiobjective optimization of IMRT plans for prostate cancer. Radiat Oncol. 2016; 11: 38. | |
dc.identifier.citedreference | Mayo CS, Moran JM, Bosch W, et al. AAPM TG‐263: standardizing nomenclatures in radiation oncology. Int J Radiat Oncol Biol Phys. 2017; 99: E552. | |
dc.identifier.citedreference | Breiman L. Statistical modeling: the two cultures (with comments and a rejoinder by the author). Statist Sci. 2001; 16: 199 – 231. | |
dc.identifier.citedreference | Dekker A, Vinod S, Holloway L, et al. Rapid learning in practice: a lung cancer survival decision support system in routine patient care data. Radiother Oncol. 2014; 113: 47 – 53. | |
dc.identifier.citedreference | Figueroa RL, Zeng‐Treitler Q, Kandula S, Ngo LH. Predicting sample size required for classification performance. BMC Med Inform Decis Mak. 2012; 12: 8 – 8. | |
dc.identifier.citedreference | Wu Y. Elastic net for cox’s proportional hazards model with a solution path algorithm. Stat Sin. 2012; 22: 27 – 294. | |
dc.identifier.citedreference | Japkowicz N, Shah M. Performance evaluation in machine learning. In: El Naqa I, Li R, Murphy MJ, eds. Machine Learning in Radiation Oncology: Theory and Applications. Switzerland: Springer‐Verlag; 2015: 41 – 56. | |
dc.identifier.citedreference | Page P. Beyond statistical significance: clinical interpretation of rehabilitation research literature. Int J Sports Phys Ther. 2014; 9: 726 – 736. | |
dc.identifier.citedreference | Harris RF. Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Billions. New York: Basic Books; 2017. | |
dc.identifier.citedreference | Hastie T, Tibshirani R, Friedman JH. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edn. New York, NY: Springer; 2009. | |
dc.identifier.citedreference | Vallieres M, Freeman CR, Skamene SR, El Naqa I. A radiomics model from joint FDG‐PET and MRI texture features for the prediction of lung metastases in soft‐tissue sarcomas of the extremities. Phys Med Biol. 2015; 60: 5471 – 5496. | |
dc.identifier.citedreference | Samuel AL. Some studies in machine learning using the game of checkers. IBM J Res Dev. 1959; 3: 210 – 229. | |
dc.identifier.citedreference | Mitchell TM. Machine Learning. New York, NY: McGraw‐Hill; 1997. | |
dc.identifier.citedreference | Alpaydin E. Introduction to Machine Learning. 3rd ed. Cambridge, MA: The MIT Press; 2014. | |
dc.identifier.citedreference | Bishop CM. Pattern Recognition and Machine Learning. New York, NY: Springer; 2006. | |
dc.identifier.citedreference | Apolloni B. Machine Learning and Robot Perception. Berlin: Springer‐Verlag; 2005. | |
dc.identifier.citedreference | Ao S‐I, Rieger BB, Amouzegar MA. Machine Learning and Systems Engineering. Dordrecht, NY: Springer; 2010. | |
dc.identifier.citedreference | Yang ZR. Machine Learning Approaches to Bioinformatics. Hackensack, NJ: World Scientific; 2010. | |
dc.identifier.citedreference | Cleophas TJ. Machine Learning in Medicine. New York, NY: Springer; 2013. | |
dc.identifier.citedreference | Malley JD, Malley KG, Pajevic S. Statistical Learning for Biomedical Data. Cambridge: Cambridge University Press; 2011. | |
dc.identifier.citedreference | Friedman J, Hastie T, Tibshirani R. The Elements of Statistical Learning. Vol 1: Berlin: Springer series in statistics Springer; 2001. | |
dc.identifier.citedreference | Lustberg T, van Soest J, Gooding M, et al. Clinical evaluation of atlas and deep learning based automatic contouring for lung cancer. Radiother Oncol. 2018; 126: 312 – 317. | |
dc.identifier.citedreference | Chapelle O, Schlkopf B, Zien A. Semi‐Supervised Learning. Cambridge, MA: The MIT Press; 2010. | |
dc.identifier.citedreference | Park SH, Gao Y, Shi Y, Shen D. Interactive prostate segmentation using atlas‐guided semi‐supervised learning and adaptive feature selection. Med Phys. 2014; 41: 111715. | |
dc.identifier.citedreference | Soares I, Dias J, Rocha H, Khouri L, Do Carmo Lopes M, Ferreira B. Semi‐supervised self‐training approaches in small and unbalanced datasets: application to xerostomia radiation side‐effect. In Kyriacou E, Christofides S, Pattichis CS, eds. XIV Mediterranean Conference on Medical and Biological Engineering and Computing 2016: MEDICON 2016, March 31st‐April 2nd 2016, Paphos, Cyprus. Cham: Springer International Publishing; 2016: 828 – 833. | |
dc.identifier.citedreference | Collins GS, Reitsma JB, Altman DG, Moons KM. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (tripod): the tripod statement. Ann Intern Med. 2015; 162: 55 – 63. | |
dc.identifier.citedreference | Luo W, Phung D, Tran T, et al. Guidelines for developing and reporting machine learning predictive models in biomedical research: a multidisciplinary view. J Med Internet Res. 2016; 18: e323. | |
dc.identifier.citedreference | Willoughby TR, Starkschall G, Janjan NA, Rosen II. Evaluation and scoring of radiotherapy treatment plans using an artificial neural network. Int J Radiat Oncol Biol Phys. 1996; 34: 923 – 930. | |
dc.identifier.citedreference | Rowbottom CG, Webb S, Oldham M. Beam‐orientation customization using an artificial neural network. Phys Med Biol 1999; 44: 2251 – 2262. | |
dc.identifier.citedreference | Wells DM, Niederer J. A medical expert system approach using artificial neural networks for standardized treatment planning. Int J Radiat Oncol Biol Phys 1998; 41: 173 – 182. | |
dc.identifier.citedreference | Gulliford SL, Webb S, Rowbottom CG, Corne DW, Dearnaley DP. Use of artificial neural networks to predict biological outcomes for patients receiving radical radiotherapy of the prostate. Radiother Oncol. 2004; 71: 3 – 12. | |
dc.identifier.citedreference | Munley MT, Lo JY, Sibley GS, Bentel GC, Anscher MS, Marks LB. A neural network to predict symptomatic lung injury. Phys Med Biol. 1999; 44: 2241 – 2249. | |
dc.identifier.citedreference | Su M, Miften M, Whiddon C, Sun X, Light K, Marks L. An artificial neural network for predicting the incidence of radiation pneumonitis. Med Phys. 2005; 32: 318 – 325. | |
dc.identifier.citedreference | NAqA IE, Deasy JO, Mu Y, et al. Datamining approaches for modeling tumor control probability. Acta Oncol. 2010; 49: 1363 – 1373. | |
dc.identifier.citedreference | Valdes G, Solberg TD, Heskel M, Ungar L, Simone CB II. Using machine learning to predict radiation pneumonitis in patients with stage I non‐small cell lung cancer treated with stereotactic body radiation therapy. Phys Med Biol. 2016; 61: 6105. | |
dc.identifier.citedreference | El Naqa I, Bradley J, Blanco AI, et al. Multivariable modeling of radiotherapy outcomes, including dose–volume and clinical factors. Int J Radiat Oncol Biol Phys. 2006; 64: 1275 – 1286. | |
dc.identifier.citedreference | Oberije C, Nalbantov G, Dekker A, et al. A prospective study comparing the predictions of doctors versus models for treatment outcome of lung cancer patients: a step toward individualized care and shared decision making. Radiother Oncol. 2014; 112: 37 – 43. | |
dc.identifier.citedreference | Bradley J, Deasy JO, Bentzen S, El Naqa I. Dosimetric correlates for acute esophagitis in patients treated with radiotherapy for lung carcinoma. Int J Radiat Oncol Biol Phys. 2004; 58: 1106 – 1113. | |
dc.identifier.citedreference | Hope AJ, Lindsay PE, El Naqa I, et al. Modeling radiation pneumonitis risk with clinical, dosimetric, and spatial parameters. Int J Radiat Oncol Biol Phys. 2006; 65: 112 – 124. | |
dc.identifier.citedreference | Zhou Z, Folkert M, Cannon N, et al. Predicting distant failure in early stage NSCLC treated with SBRT using clinical parameters. Radiother Oncol. 2016; 119: 501 – 504. | |
dc.identifier.citedreference | Kalet AM, Gennari JH, Ford EC, Phillips MH. Bayesian network models for error detection in radiotherapy plans. Phys Med Biol. 2015; 60: 2735 – 2749. | |
dc.identifier.citedreference | Valdes G, Scheuermann R, Hung C, Olszanski A, Bellerive M, Solberg T. A mathematical framework for virtual IMRT QA using machine learning. Med Phys. 2016; 43: 4323 – 4334. | |
dc.identifier.citedreference | Valdes G, Morin O, Valenciaga Y, Kirby N, Pouliot J, Chuang C. Use of TrueBeam developer mode for imaging QA. J Appl Clin Med Phys. 2015; 16: 322 – 333. | |
dc.identifier.citedreference | Carlson JN, Park JM, Park S‐Y, Park JI, Choi Y, Ye S‐J. A machine learning approach to the accurate prediction of multi‐leaf collimator positional errors. Phys Med Biol. 2016; 61: 2514. | |
dc.identifier.citedreference | Li Q, Chan MF. Predictive time‐series modeling using artificial neural networks for Linac beam symmetry: an empirical study. Ann N Y Acad Sci. 2017; 1387: 84 – 94. | |
dc.identifier.citedreference | Moore KL, Brame RS, Low DA, Mutic S. Experience‐based quality control of clinical intensity‐modulated radiotherapy planning. Int J Radiat Oncol Biol Phys. 2011; 81: 545 – 551. | |
dc.identifier.citedreference | Wu B, Ricchetti F, Sanguineti G, et al. Data‐driven approach to generating achievable dose‐volume histogram objectives in intensity‐modulated radiotherapy planning. Int J Radiat Oncol Biol Phys. 2011; 79: 1241 – 1247. | |
dc.identifier.citedreference | Wu B, Ricchetti F, Sanguineti G, et al. Patient geometry‐driven information retrieval for IMRT treatment plan quality control. Med Phys. 2009; 36: 5497 – 5505. | |
dc.identifier.citedreference | Guidi G, Maffei N, Meduri B, et al. A machine learning tool for re‐planning and adaptive RT: a multicenter cohort investigation. Physica Med. 2016; 32: 1659 – 1666. | |
dc.identifier.citedreference | Guidi G, Maffei N, Vecchi C, et al. A support vector machine tool for adaptive tomotherapy treatments: prediction of head and neck patients criticalities. Physica Med. 2015; 31: 442 – 451. | |
dc.identifier.citedreference | Ruan D, Keall P. Online prediction of respiratory motion: multidimensional processing with low‐dimensional feature learning. Phys Med Biol. 2010; 55: 3011 – 3025. | |
dc.identifier.citedreference | Isaksson M, Jalden J, Murphy MJ. On using an adaptive neural network to predict lung tumor motion during respiration for radiotherapy applications. Med Phys. 2005; 32: 3801 – 3809. | |
dc.identifier.citedreference | El Naqa I, Li R, Murphy MJ, eds. Machine Learning in Radiation Oncology: Theory and Application. Switzerland: Springer International Publishing; 2015. https://doi.org/10.1007/978-3-319-18305-3. | |
dc.identifier.citedreference | Kohannim O, Hua X, Hibar DP, et al. Boosting power for clinical trials using classifiers based on multiple biomarkers. Neurobiol Aging. 2010; 31: 1429 – 1442. | |
dc.identifier.citedreference | Head ML, Holman L, Lanfear R, Kahn AT, Jennions MD. The extent and consequences of P‐hacking in science. PLoS Biol. 2015; 13: e1002106. | |
dc.identifier.citedreference | Smith WP, Phillips MH. Comment on “ROC analysis in patient specific quality assurance” [Med. Phys. 40(4), 042103 (7 pp.) (2013)]. Med Phys. 2015; 42: 4411 – 4412. | |
dc.identifier.citedreference | Hand DJ. Measuring classifier performance: a coherent alternative to the area under the ROC curve. Mach Learn. 2009; 77: 103 – 123. | |
dc.identifier.citedreference | Phillips MH, Smith WP, Parvathaneni U, Laramore GE. Role of positron emission tomography in the treatment of occult disease in head‐and‐neck cancer: a modeling approach. Int J Radiat Oncol Biol Phys. 2011; 79: 1089 – 1095. | |
dc.identifier.citedreference | Halligan S, Altman DG, Mallett S. Disadvantages of using the area under the receiver operating characteristic curve to assess imaging tests: a discussion and proposal for an alternative approach. Eur Radiol. 2015; 25: 932 – 939. | |
dc.identifier.citedreference | Rice ME, Harris GT. Comparing effect sizes in follow‐up studies: ROC Area, Cohen’s d, and r. Law Hum Behav. 2005; 29: 615 – 620. | |
dc.identifier.citedreference | Kraemer HC, Kupfer DJ. Size of treatment effects and their importance to clinical research and practice. Biol Psychiat. 2006; 59: 990 – 996. | |
dc.owningcollname | Interdisciplinary and Peer-Reviewed |
Files in this item
Remediation of Harmful Language
The University of Michigan Library aims to describe library materials in a way that respects the people and communities who create, use, and are represented in our collections. Report harmful or offensive language in catalog records, finding aids, or elsewhere in our collections anonymously through our metadata feedback form. More information at Remediation of Harmful Language.
Accessibility
If you are unable to use this file in its current format, please select the Contact Us link and we can modify it to make it more accessible to you.