Radiotherapy Treatment Plan Optimization and Decision Making using Advanced Machine Learning Approaches

Abstract

The advancement of radiation therapy (RT) over the years has greatly improved patient outcomes and quality of life and RT remains a standard component of care for many cancer patients. Proper delivery of RT requires meticulous treatment planning to maximize the dose to the tumor and minimize the dose to surrounding healthy tissues. Although there are similarities in anatomical structures between patients for a specific disease site, there is still anatomic variation and a one-size-fits-all approach to treatment planning is insufficient. In current clinical practice, a planner iteratively tunes cost function parameters in a trial-and-error fashion to find an optimal plan. This means that plan quality is highly dependent on the skills and experience of the individual planner and their time availability to attempt different parameter configurations to achieve a clinically acceptable plan. This dissertation investigates advanced machine learning approaches to address these challenges in RT treatment planning optimization. Ultimately, treatment planning optimization is a human decision-making problem, which current automation methods do not adequately capture. We developed a dosimetry-guided cooperative multi-agent reinforcement learning (MARL) framework to optimize the tuning weights of organs at risk (OARs) in stereotactic body radiation therapy (SBRT) central lung cancer cases, combining deep learning and knowledge-based planning (KBP)—through a novel human-integrated training approach—into a single framework. MARL generates a plan in under 30 minutes, improving optimization time up to 83%. These plans are quantitatively comparable to clinical plans, reporting differences of 0.01–0.02, 0.001–0.12, 0.30–0.11, and 0.01 for Paddick Conformity Index (PCI), Homogeneity Index (HI), R50 Gradient Index (R50), and our own plan scoring metric (PSM/PSMMax), respectively. MARL significantly reduced the dose to agent-assigned OARs, generating clinically acceptable plans. Our domain expert evaluations indicate physicians’ preference influences the approval of the final treatment plan even when they meet clinical objectives. These results suggest a potential for MARL to be utilized as a high-quality starting point in treatment planning to improve planning time efficiency and reduce plan quality variability. To further improve MARL, we considered quantum-inspired algorithms. Studies show quantum probability theory can model the human decision-making process. We hypothesized that leveraging the intrinsic probabilistic nature of qubits would better capture the uncertainty of planner intent during treatment planning optimization, which is not defined through conventional reinforcement learning (RL) methods. A quantum approach to RL also offers better explainable exploration-exploitation trade-off and improved learning performance in larger state-action space. We first developed and tested a single-agent quantum reinforcement learning (QRL) framework to predict dose adjustments for adaptive radiotherapy treatment designed as a clinical decision support system. The quantum algorithm dose suggestions showed a 10% improvement over clinical dose decisions. Following this success, we incorporated the quantum algorithm into MARL to develop a q-MARL framework. The action space was extended to also adjust target dose values alongside the objective weights, taking advantage of the quantum representation for the action states. q-MARL plans are quantitatively comparable to both MARL and clinical plans, with differences of 0.01–0.02, 0.07–0.23, 0.02–0.23, and 0.01 for PCI, HI, R50, and PSM/PSMMax, respectively. Compared to MARL, the q-MARL plans showed better sparing of OARs, especially for ones that overlapped the planning target volume (PTV). The work in this dissertation demonstrates the feasibility of applying quantum and multi-agent reinforcement learning as a joint framework to assist human planners via workflow standardization and automation for radiation therapy.