Data, Model, Inference: Evaluating Glioma MRI Models on Clinically Informed Axes

Abstract

Background: Machine learning algorithms have outgrown our ability to understand them, and are used in many settings, though less frequently in clinical practice. Magnetic resonance imaging (MRI) of gliomas and prediction of genetic features serve as a useful machine learning problems for developing evaluation tools for clinical algorithms. Gliomas and particularly glioma classification are clinically important because of the wide range of outcomes for different glioma subtypes. Glioblastomas have a median survival of 15 months with treatment, whereas oligodendrogliomas with 1p/19q codeletion have an 80% 15-year survival rate with treatment. Genetic markers such as IDH mutation status, MGMT promoter methylation, and 1p/19q codeletion have been shown to have important prognostic or treatment ramifications. As a result, World Health Organization (WHO) guidelines have changed to showcase the importance of these markers in the classification of gliomas, sometimes overriding traditional histopathological grading. This dissertation uses models built on different parts of the glioma genetic prediction pipeline. It then investigates the data that built the models, the robustness of the models that make the predictions, and the inferences gained from the model predictions.

Methods: The first set of experiments tests a variety of imaging features to predict the presence of 1p/19q codeletion. These features were then judged by robustness to image and mask perturbation, before filtering out the non-robust features. The next work presented covers glioma segmentation models, and evaluates models built on different data subsets to assess multiple dimensions of the models. Models were assessed on extensive metrics, model confidence, out of distribution performance, robustness to adversarial attack, and correlations with data quality metrics. Later, MRI artifacts in acquisition and preprocessing were developed and simulated to test a tumor segmentation for robustness to real world failure modes. Using MRI physics, literature guidance, and clinical expertise, a toolbox for assessing glioma segmentation models was developed with tunable artifacts for testing models until failure. Lastly, model inference was tested using contralateral and ipsilateral radiomic features to attempt to predict genetic markers.

Results: Texture/shape features were particularly susceptible to volume alteration perturbations, and other feature sets such as topological or neural network features were more robust. 1p/19q codeletion models based on robust features had similar performance to those without filtering. Models built on the full dataset rather than subsets had more consistent performance with model confidence and didn’t fail as quickly under adversarial attack as glioblastoma trained models. MRI quality metrics were found to vary across sites and data collections but weren’t correlated with these models’ performances. Seven different varieties of MRI artifacts were successfully simulated, and motion and field inhomogeneity had the most significant impact of the acquisition artifacts. Among preprocessing artifacts, sequence mislabeling could have drastic impacts on model performance, and sequence misalignment also had significant effects. Contralateral radiomic features were found to be as predictive of IDH mutation status as tumor features, and were also predictive for TERT and ATRX, though not predictive of MGMT. Spherical core samples had some impact on reducing the power of the contralateral, but predictive power was sometimes still significant. Care should be taken in using the contralateral, and more investigation should be done into potential signals of tumor invasion in the contralateral brain. Overall, medical imaging models should be tested in many ways to improve our understanding and trust of these complex pipelines.