Abstract:Cancer is one of the most life-threatening diseases worldwide, and head and neck (H&N) cancer is a prevalent type with hundreds of thousands of new cases recorded each year. Clinicians use medical imaging modalities such as computed tomography and positron emission tomography to detect the presence of a tumor, and they combine that information with clinical data for patient prognosis. The process is mostly challenging and time-consuming. Machine learning and deep learning can automate these tasks to help clinicians with highly promising results. This work studies two approaches for H&N tumor segmentation: (i) exploration and comparison of vision transformer (ViT)-based and convolutional neural network-based models; and (ii) proposal of a novel 2D perspective to working with 3D data. Furthermore, this work proposes two new architectures for the prognosis task. An ensemble of several models predicts patient outcomes (which won the HECKTOR 2021 challenge prognosis task), and a ViT-based framework concurrently performs patient outcome prediction and tumor segmentation, which outperforms the ensemble model.

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are the automated diagnosis and prognosis of head and neck cancer (H&N cancer), specifically including: 1. **Head and Neck Cancer Tumor Segmentation**: - The paper studied two methods for head and neck cancer tumor segmentation: 1. Compare the effects of Vision Transformer (ViT) - based and Convolutional Neural Network (CNN) - based models. 2. Propose a new 2D perspective to process 3D medical image data by generating "Super Images" to simplify the processing of 3D data. 2. **Patient Prognosis Prediction**: - The paper proposed two new architectures for the patient prognosis prediction task: 1. A method of integrating multiple models that can simultaneously extract features and generate risk scores for predicting patients' survival time. This integrated model won the prognosis task of the HECKTOR 2021 challenge. 2. A ViT - based framework that can simultaneously perform patient prognosis prediction and tumor segmentation and outperforms the integrated model in performance. ### Specific Problem Description - **Tumor Segmentation Problem**: Traditional head and neck cancer tumor segmentation mainly relies on CNN models, especially 3D networks. However, ViT - based models have not been widely studied, and there is no 2D method to process 3D medical image data. Therefore, the paper explored the potential of ViT models in tumor segmentation and proposed a new 2D processing method. - **Prognosis Prediction Problem**: In the existing literature, the prognosis prediction of head and neck cancer patients usually uses traditional machine learning models or deep learning models for feature extraction and then applies traditional machine learning models. However, there is no integrated model that can simultaneously analyze imaging and Electronic Health Record (EHR) data and predict patient prognosis. In addition, the potential of ViT models in multi - modal data input has not been fully explored. ### Solutions - **Tumor Segmentation**: - Use ViT models to compare with the existing leading CNN models and evaluate their performance in the tumor segmentation task. - Propose a new 2D perspective method, which simplifies the processing flow and improves efficiency by converting 3D data into 2D super - images. - **Prognosis Prediction**: - Propose an integrated model that combines multiple deep learning and machine learning models to improve the accuracy of prognosis prediction. - Design a ViT - based framework that can simultaneously process imaging and EHR data to achieve more comprehensive prognosis prediction and tumor segmentation. Through these methods, the paper aims to improve the automation level of diagnosis and prognosis prediction for head and neck cancer patients, thereby helping clinicians make more efficient and accurate diagnosis and treatment decisions.

Diagnosis and Prognosis of Head and Neck Cancer Patients using Artificial Intelligence

An Ensemble Approach for Patient Prognosis of Head and Neck Tumor Using Multimodal Data

Artificial Intelligence-based methods in head and neck cancer diagnosis: an overview

A Machine Learning Challenge for Prognostic Modelling in Head and Neck Cancer Using Multi-modal Data

Towards Tumour Graph Learning for Survival Prediction in Head & Neck Cancer Patients

Multi-institutional Prognostic Modeling in Head and Neck Cancer: Evaluating Impact and Generalizability of Deep Learning and Radiomics

Artificial Intelligence in Head and Neck Cancer Diagnosis: A Comprehensive Review with Emphasis on Radiomics, Histopathological, and Molecular Applications

Multi-task Deep Learning for Joint Tumor Segmentation and Outcome Prediction in Head and Neck Cancer.

Segmentation-Free Outcome Prediction in Head and Neck Cancer: Deep Learning-based Feature Extraction from Multi-Angle Maximum Intensity Projections (MA-MIPs) of PET Images

Artificial Intelligence in Head and Neck Cancer: Innovations, Applications, and Future Directions

A generalized AI method for pathology cancer diagnosis and prognosis prediction based on transfer learning and hierarchical split

An Innovative and Efficient Diagnostic Prediction Flow for Head and Neck Cancer: A Deep Learning Approach for Multi-Modal Survival Analysis Prediction Based on Text and Multi-Center PET/CT Images

Artificial intelligence in cancer imaging: Clinical challenges and applications

Recent Advances in Deep Learning and Medical Imaging for Head and Neck Cancer Treatment: MRI, CT, and PET Scans

Enhancing head and neck tumor management with artificial intelligence: Integration and perspectives

Artificial Intelligence-Driven radiomics in head and neck Cancer: Current status and future prospects

Current status and prospects of artificial intelligence in breast cancer pathology: convolutional neural networks to prospective Vision Transformers

Application of Deep Learning in Cancer Prognosis Prediction Model

Deep Learning Based Time-to-event Analysis with PET, CT and Joint PET/CT for Head and Neck Cancer Prognosis

Predicting cancer outcomes from histology and genomics using convolutional networks

Role of Artificial Intelligence and Machine Learning in Prediction, Diagnosis, and Prognosis of Cancer