Abstract:Pulmonary cancer is one of the most common malignancies worldwide. Accurate classification of its subtypes is required in differential diagnosis. However, existing algorithms are mostly based on color images and the improvement of accuracy is quite challenging. In this study, we propose a Convolution Combination Unit (CCU) based three-dimensional Convolutional Neural Network (3D-PulCNN) for classifying pulmonary cancer presented in microscopic hyperspectral image with both spatial and spectral information. CCU is designed to fuse the features acquired by different convolution scales. Compared with VGGNet, only two fully connected layers are used in this model, reducing the network parameters and model complexity. Experimental results show that 3D-PulCNN achieves Overall Average (OA) of 0.962 and Precision, Recall and Kappa of more than 0.920, superior to 2D-VGGNet. Then 3D-UNet is leveraged to segment cancer cells and their morphological characteristics are calculated to supply quantitative virtual analysis data for classification results explanation and prognosis assessment.This article is protected by copyright. All rights reserved.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the accurate classification of lung cancer subtypes. Specifically, the authors propose a three - dimensional convolutional neural network (3D - PulCNN) based on the convolutional combination unit (CCU) for classifying lung cancer from microscopic hyperspectral images. Most traditional methods are based on color images, and these methods have encountered great challenges in improving classification accuracy. By utilizing the spatial and spectral information in microscopic hyperspectral images, this paper aims to provide a more accurate method for classifying lung cancer. ### Background Lung cancer is one of the most common malignant tumors in the world, and the accurate classification of its subtypes is crucial for clinical diagnosis. However, existing algorithms mainly rely on color images, which makes it very difficult to improve accuracy. This paper proposes a new method, 3D - PulCNN, which combines convolutional features at different scales to fuse spatial and spectral information, thereby improving the accuracy of classification. ### Method 1. **Data set**: - The data set is obtained through an independently developed acousto - optic tunable filter (AOTF) microscopic hyperspectral imaging system and contains lung cancer samples in 150 scenes (50 scenes for each subtype). - Each sample is stained with HE and collected in the original format (HSI) and jpg format (RGB image). 2. **Data pre - processing**: - Use the Lambert - Beer law to remove the influence of system noise. - Use principal component analysis (PCA) to remove low - quality bands and extract the first 8 principal components. 3. **Classification algorithm**: - A 3D - PulCNN model is proposed, which contains a convolutional combination unit (CCU) for simultaneously extracting spatial and spectral features. - The model structure includes one CCU layer, four convolutional layers, five pooling layers and two fully - connected layers. - Evaluate the model through the training set, validation set and test set, with the ratios of 6:2:2 respectively. 4. **Quantitative virtual analysis**: - Use 3D - UNet to segment cancer cells of different subtypes and calculate cell morphological parameters, such as cell area (CA), cell perimeter (CP), cell roundness (CC), cell major axis (CMAA), cell minor axis (CMIA) and cell eccentricity (CE). - These parameters are helpful for pathologists to conduct quantitative research, further explain the classification results and evaluate the prognosis. ### Results - 3D - PulCNN has reached an overall average accuracy (OA) of 0.962, and the precision, recall and Kappa coefficient all exceed 0.920, which is better than 2D - VGGNet. - Compared with other methods such as VGGNet, 3D SE - ResNet, 2D - CNN and 3D - CNN, 3D - PulCNN shows significant advantages in multiple performance indicators. - By segmenting cancer cells with 3D - UNet and calculating cell morphological parameters, the effectiveness and practicality of the model are further verified. ### Conclusion The 3D - PulCNN model proposed in this paper shows high accuracy and robustness in the classification of lung cancer subtypes. By combining the spatial and spectral information of microscopic hyperspectral images, this model can effectively improve the accuracy of classification and provide strong support for the diagnosis and treatment of lung cancer.

3D‐PulCNN : Pulmonary cancer classification from hyperspectral images using convolution combination unit based CNN

A Hybrid CNN Feature Model for Pulmonary Nodule Malignancy Risk Differentiation.

Multimodality Attention-Guided 3-D Detection of Nonsmall Cell Lung Cancer in 18 F-FDG PET/CT Images

Pulmonary nodule classification in lung cancer screening with three-dimensional convolutional neural networks

3D Convolutional Neural Network for Differentiating Pre-Invasive Lesions from Invasive Adenocarcinomas Appearing As Ground-Glass Nodules with Diameters ≤3 Cm Using HRCT

Identification of Pulmonary Nodules Via CT Images with Hierarchical Fully Convolutional Networks.

Classification Of Lung Nodule Malignancy Risk On Computed Tomography Images Using Convolutional Neural Network: A Comparison Between 2d And 3d Strategies

An improved 3-D attention CNN with hybrid loss and feature fusion for pulmonary nodule classification

Multi-Layer Perceptron Classifier with the Proposed Combined Feature Vector of 3D CNN Features and Lung Radiomics Features for COPD Stage Classification

Lung Cancer Classification Using Image Enhancement and CNN-Based Pre-Trained Models

Classification of Pulmonary Nodules in 2-[18F]FDG PET/CT Images with a 3D Convolutional Neural Network

3D Multi-View Squeeze-and-excitation Convolutional Neural Network for Lung Nodule Classification

Toward an Expert Level of Lung Cancer Detection and Classification Using a Deep Convolutional Neural Network.

Risk Stratification of Lung Nodules Using 3D CNN-Based Multi-task Learning

Automated Pulmonary Nodule Detection in CT Images Using Deep Convolutional Neural Networks.

Enhancing Lung Cancer Survival Prediction: 3D CNN Analysis of CT Images Using Novel GTV1-SliceNum Feature and PEN-BCE Loss Function

Classification of lung cancer computed tomography images using a 3-dimensional deep convolutional neural network with multi-layer filter

Novel convolutional neural network architecture for improved pulmonary nodule classification on computed tomography

Deep 3D Convolutional Neural Network for Automated Lung Cancer Diagnosis

Combination of Generative Adversarial Network and Convolutional Neural Network for Automatic Subcentimeter Pulmonary Adenocarcinoma Classification.

High-risk Factor Prediction in Lung Cancer Using Thin CT Scans: An Attention-Enhanced Graph Convolutional Network Approach