Abstract:Background Treatment planning is currently a patient specific, time‐consuming, and resource demanding task in radiotherapy. Dose‐volume histogram (DVH) prediction plays a critical role in automating this process. The geometric relationship between DVHs in radiotherapy plans and organs‐at‐risk (OAR) and planning target volume (PTV) has been well established. This study explores the potential of deep learning models for predicting DVHs using images and subsequent human intervention facilitated by a large‐language model (LLM) to enhance the planning quality. Method We propose a pipeline to convert unstructured images to a structured graph consisting of image‐patch nodes and dose nodes. A novel Dose Graph Neural Network (DoseGNN) model is developed for predicting DVHs from the structured graph. The proposed DoseGNN is enhanced with the LLM to encode massive knowledge from prescriptions and interactive instructions from clinicians. In this study, we introduced an online human‐AI collaboration (OHAC) system as a practical implementation of the concept proposed for the automation of intensity‐modulated radiotherapy (IMRT) planning. Results The proposed DoseGNN model was compared to widely employed DL models used in radiotherapy, including Swin Transformer, 3D U‐Net CNN, and vanilla MLP. For PTV, DoseGNN achieved the mean absolute error (MAE) of Dmax , Dmean , D95 , and D1 between true plans and predicted plans that were 64%, 53%, 64%, 61% of the best baseline model. For the worst case among OARs (left lung, right lung, chest wall, heart, spinal cord), DoseGNN achieved the mean absolute error of Dmax , Dmean , D50 that were 85%, 91%, 80% of the best baseline model. Moreover, the LLM‐empowered DoseGNN model facilitates seamless adjustment to treatment plans through interaction with clinicians using natural language. Conclusion We developed DoseGNN, a novel deep learning model for predicting delivered radiation doses from medical images, enhanced by LLM to allow adjustment through seamless interaction with clinicians. The preliminary results confirm DoseGNN's superior accuracy in DVH prediction relative to typical DL methods, highlighting its potential to facilitate an online clinician‐AI collaboration system for streamlined treatment planning automation.

A method of using deep learning to predict three‐dimensional dose distributions for intensity‐modulated radiotherapy of rectal cancer

Dose distributions prediction for intensity-modulated radiotherapy of postoperative rectal cancer based on deep learning

Automatic Treatment Planning Based on Three-Dimensional Dose Distribution Predicted from Deep Learning Technique.

A New Deep-Learning-based Model for Predicting 3D Radiotherapy Dose Distribution In Various Scenarios

A Feasibility Study for Predicting 3D Radiotherapy Dose Distribution of Lung VMAT Patients

Large‐language‐model empowered 3D dose prediction for intensity‐modulated radiotherapy

A Feasibility Study on Deep Learning Based Individualized 3D Dose Distribution Prediction

AS-NeSt: a Novel 3D Deep Learning Model for Radiotherapy Dose Distribution Prediction on Esophageal Cancer Treatment with Multiple Prescriptions

AS-NeSt: A Novel 3D Deep Learning Model for Radiation Therapy Dose Distribution Prediction in Esophageal Cancer Treatment with Multiple Prescriptions

A feasibility study for predicting optimal radiation therapy dose distributions of prostate cancer patients from patient anatomy using deep learning

Three-dimensional dose prediction based on deep convolutional neural networks for brain cancer in CyberKnife: accurate beam modelling of homogeneous tissue

A Comparative Study of Deep Learning Dose Prediction Models for Cervical Cancer Volumetric Modulated Arc Therapy

Deep learning architecture with shunted transformer and 3D deformable convolution for voxel-level dose prediction of head and neck tumors

Data-driven dose calculation algorithm based on deep learning

DiffDP: Radiotherapy Dose Prediction via a Diffusion Model

A Feasibility Study on Deep Learning-Based Radiotherapy Dose Calculation

Deep learning–based dose prediction to improve the plan quality of volumetric modulated arc therapy for gynecologic cancers

Technical Note: A feasibility study on deep learning-based radiotherapy dose calculation

Direct Dose Prediction With Deep Learning for Postoperative Cervical Cancer Underwent Volumetric Modulated Arc Therapy

Deep Learning 3D Dose Prediction for Conventional Lung IMRT Using Consistent/Unbiased Automated Plans

Dose Prediction Driven Radiotherapy Paramters Regression via Intra- and Inter-Relation Modeling