Advancing Clinical Outcome Prediction through Innovative Multimodal and Domain-Generalized AI that Accommodates Limited Data

Abstract

Clinical decision support systems are computer-based systems developed with the goal of assisting health care providers in arduous clinical tasks or improving decision-making. In routine clinical care, medical practices tend to be dynamic and must account for diversity of data. In this thesis, we focus on developing innovative multimodal and multidomain AI models for clinical decision support, with a focus on applications with limited data availability. We start with a survey chapter followed by three case studies of multimodal/multidomain proof-of-concept Clinical Decision Support (CDS) models that accommodate limited data. Our research seeks to address questions regarding constructing machine-learning-based models that mimic real-world mental models and bridge domain gaps in cases of limited data. In the first chapter, we explore a survey of state-of-the-art methods in multimodal machine learning applied to biomedicine, highlighting how these models address five challenges of multimodal machine learning: representation, fusion, translation, alignment and co-learning. Next, we tackle a case study where we develop a low-parameter model to discriminate pseudoprogression and true progression in glioblastoma using a small sample of MRI images. Then, we develop a clinically-informed privileged learning model which leverages both routine clinical data and privileged information (CBCT and protein serum/saliva tests) to detect Temporomandibular Joint Osteoarthritis (TMJ OA). Finally, we present a case of domain generalization to allow a model trained on one Alcon SN60WF lens to predict post-operative refraction in patients implanted with other lenses in cataract surgery, with an attempt to adapt to other populations and “A-constants” as well. We present these three case studies as examples of informed models that accommodate diverse data types, as real-world clinical practice is intrinsically multimodal and multidomain. We hope these models provide inspiration for additional models outside of the provided use cases and assert that methodologies can be combined and adapted as needed.