Abstract:Liver tumor segmentation and classification are important tasks in computer aided diagnosis. We aim to address three problems: liver tumor screening and preliminary diagnosis in non-contrast computed tomography (CT), and differential diagnosis in dynamic contrast-enhanced CT. A novel framework named Pixel-Lesion-pAtient Network (PLAN) is proposed. It uses a mask transformer to jointly segment and classify each lesion with improved anchor queries and a foreground-enhanced sampling loss. It also has an image-wise classifier to effectively aggregate global information and predict patient-level diagnosis. A large-scale multi-phase dataset is collected containing 939 tumor patients and 810 normal subjects. 4010 tumor instances of eight types are extensively annotated. On the non-contrast tumor screening task, PLAN achieves 95% and 96% in patient-level sensitivity and specificity. On contrast-enhanced CT, our lesion-level detection precision, recall, and classification accuracy are 92%, 89%, and 86%, outperforming widely used CNN and transformers for lesion segmentation. We also conduct a reader study on a holdout set of 250 cases. PLAN is on par with a senior human radiologist, showing the clinical significance of our results.

What problem does this paper attempt to address?

The problems that this paper attempts to solve mainly include three aspects: 1. **Liver tumor screening and preliminary diagnosis under non - contrast - enhanced CT**: Detect and preliminarily classify liver tumors through CT images without using contrast agents. This is a challenging task because the contrast between tumors and normal tissues in non - contrast - enhanced CT images is low, making tumor detection more difficult. 2. **Differential diagnosis of liver tumors under dynamic contrast - enhanced CT**: More precisely classify liver tumors in dynamic CT images using contrast agents. Dynamic contrast - enhanced CT can provide more diagnostic information, such as the vascular characteristics of lesions and the enhancement pattern of contrast agents, which helps to distinguish different types of liver tumors. 3. **Comprehensively solve the above two problems**: Propose a new framework named Pixel - Lesion - pAtient Network (PLAN), which can simultaneously screen, preliminarily diagnose, and finely classify liver tumors under non - contrast - enhanced CT and dynamic contrast - enhanced CT. Specifically, this framework can: - Segment and classify each lesion. - Effectively aggregate global information through an image - level classifier to predict patient - level diagnosis results. ### Main contributions of the paper 1. **Dataset**: Collected a large - scale multi - phase dataset, including 939 tumor patients and 810 normal subjects, with 4,010 tumor instances labeled, involving eight different types. 2. **Model design**: Proposed an end - to - end framework PLAN, which contains three branches: - **Pixel branch**: Used to generate an initial pixel - level segmentation map. - **Lesion branch**: Based on the improved Mask Transformer, used to segment and classify each lesion. - **Patient branch**: Used to aggregate the information of the entire image, perform patient - level diagnosis, and encourage consistency with the lesion branch through a consistency loss term. 3. **Performance evaluation**: In the tumor screening task of non - contrast - enhanced CT, PLAN achieved 95.0%, 96.4%, and 0.965 in patient - level sensitivity, specificity, and average AUC respectively, significantly outperforming the widely used nnU - Net. In the lesion - level detection accuracy, recall rate, and classification accuracy of dynamic contrast - enhanced CT, they are 92.2%, 89.0%, and 85.9% respectively, also exceeding the existing methods. 4. **Clinical significance**: Through a reader study, the performance of PLAN is comparable to that of senior radiologists, showing its potential application value in clinical practice. ### Conclusion This paper successfully solved the problems of liver tumor screening, preliminary diagnosis, and fine classification under non - contrast - enhanced CT and dynamic contrast - enhanced CT by proposing the PLAN framework, demonstrating the effectiveness and clinical significance of this method in the automated screening and diagnosis of various liver tumors. Future work will further improve the specificity for refractory non - tumor cases and the sensitivity for small lesions.

Liver Tumor Screening and Diagnosis in CT with Pixel-Lesion-Patient Network

Automatic Detection of Focal Liver Lesions in Multi-phase CT Images Using A Multi-channel & Multi-scale CNN.

PA-ResSeg: A Phase Attention Residual Network for Liver Tumor Segmentation from Multi-phase CT Images

Residual Convolutional Neural Networks With Global And Local Pathways For Classification Of Focal Liver Lesions

Automatic Detection and Segmentation of Liver Tumors in Multi- Phase CT Images by Phase Attention Mask R-CNN

Improved Mask R-CNN with Deformable Convolutions for Accurate Liver Lesion Detection in Multiphase CT Images

CasCRNN-GL-Net: Cascaded Convolutional and Recurrent Neural Networks with Global and Local Pathways for Classification of Focal Liver Lesions in Multi-Phase CT Images

Tensor-Based Subspace Learning for Classification of Focal Liver Lesions in Multi-phase CT Images

Multi-Stream Scale-Insensitive Convolutional And Recurrent Neural Networks For Liver Tumor Detection In Dynamic Ct Images

Applicability of Multidimensional Convolutional Neural Networks on Automated Detection of Diverse Focal Liver Lesions in Multiphase CT Images.

A Cascaded Approach for ultraly High Performance Lesion Detection and False Positive Removal in Liver CT Scans

Attention Connect Network for Liver Tumor Segmentation from CT and MRI Images

Liver Tumor Segmentation and Classification Using FLAS-UNet++ and an Improved DenseNet

A Cascade Attention Network for Liver Lesion Classification in Weakly-Labeled Multi-phase CT Images.

LIDIA: Precise Liver Tumor Diagnosis on Multi-Phase Contrast-Enhanced CT via Iterative Fusion and Asymmetric Contrastive Learning

Fully-Automated Liver Tumor Localization and Characterization from Multi-Phase MR Volumes Using Key-Slice ROI Parsing: A Physician-Inspired Approach

Focal liver lesion diagnosis with deep learning and multistage CT imaging

APESTNet with Mask R-CNN for Liver Tumor Segmentation and Classification

Study on strategy of CT image sequence segmentation for liver and tumor based on U-Net and Bi-ConvLSTM

PA-Net: A phase attention network fusing venous and arterial phase features of CT images for liver tumor segmentation

ELTS-Net: An enhanced liver tumor segmentation network with augmented receptive field and global contextual information