Abstract:Automatic diagnosing lung cancer from Computed Tomography (CT) scans involves two steps: detect all suspicious lesions (pulmonary nodules) and evaluate the whole-lung/pulmonary malignancy. Currently, there are many studies about the first step, but few about the second step. Since the existence of nodule does not definitely indicate cancer, and the morphology of nodule has a complicated relationship with cancer, the diagnosis of lung cancer demands careful investigations on every suspicious nodule and integration of information of all nodules. We propose a 3D deep neural network to solve this problem. The model consists of two modules. The first one is a 3D region proposal network for nodule detection, which outputs all suspicious nodules for a subject. The second one selects the top five nodules based on the detection confidence, evaluates their cancer probabilities and combines them with a leaky noisy-or gate to obtain the probability of lung cancer for the subject. The two modules share the same backbone network, a modified U-net. The over-fitting caused by the shortage of training data is alleviated by training the two modules alternately. The proposed model won the first place in the Data Science Bowl 2017 competition. The code has been made publicly available.

What problem does this paper attempt to address?

This paper attempts to solve the problem of evaluating the malignancy of pulmonary nodules. Specifically, automatic diagnosis of lung cancer from computed tomography (CT) images involves two steps: detecting all suspicious lesions (pulmonary nodules) and evaluating the malignancy of the whole lung or lungs. Currently, there are more studies on the first step, but fewer on the second step. Since the presence of nodules does not necessarily mean cancer, and there is a complex relationship between the morphology of nodules and cancer, the diagnosis of lung cancer requires careful examination of each suspicious nodule and integration of the information of all nodules. To meet this challenge, the authors proposed a 3D deep neural network model. This model consists of two modules: 1. **3D Region Proposal Network (RPN)**: used to detect nodules and output all suspicious nodules. 2. **Classification module**: select the top five nodules with the highest confidence, evaluate their cancer probabilities, and combine these probabilities through a "leaky noisy - or gate" to obtain the probability of the patient having lung cancer. These two modules share the same backbone network, namely the modified U - Net. In order to alleviate the over - fitting problem caused by insufficient training data, the two modules are trained alternately. This model won the first place in the Data Science Bowl 2017 competition, and the code has been made public. ### Main contributions 1. Proposed the first one - stage end - to - end convolutional neural network (CNN) for 3D object detection. 2. Integrated the noisy - or gate into the neural network to solve the multi - instance learning (MIL) task in computer - aided diagnosis (CAD). ### Key problems solved 1. **Difficulty of 3D object detection**: 3D object detection is more difficult than 2D object detection. Directly extending 2D methods to 3D faces technical difficulties, and 3D data annotation is more difficult. 2. **Diversity of nodule morphology**: Nodules have diverse morphologies, and the differences from normal tissues are not obvious. Even experienced doctors may find it difficult to reach an agreement. 3. **Complexity of the relationship between nodules and cancer**: The presence of nodules does not necessarily mean lung cancer. For patients with multiple nodules, all nodules need to be considered comprehensively to infer the cancer probability. ### Method overview 1. **Pre - processing**: Convert CT images to Hounsfield units (HU), extract lung masks, calculate convex hulls and dilate them, and perform intensity normalization. 2. **Detection**: Use 3D RPN to detect suspicious nodules, and adopt patch - based training strategies and data augmentation techniques. 3. **Classification**: Select the top five nodules with the highest confidence and combine the cancer probabilities of these nodules through the leaky noisy - or model. ### Conclusion This study effectively solves the difficult problem of evaluating the malignancy of pulmonary nodules by proposing an innovative 3D deep - learning model, providing strong support for the early diagnosis of lung cancer.

Evaluate the Malignancy of Pulmonary Nodules Using the 3D Deep Leaky Noisy-or Network

Evaluate the Malignancy of Pulmonary Nodules Using the 3-D Deep Leaky Noisy-OR Network

Deep 3D Multi-Scale Dual Path Network for Automatic Lung Nodule Classification

DeepLung: 3D Deep Convolutional Nets for Automated Pulmonary Nodule Detection and Classification

Automated pulmonary nodule detection using 3D deep convolutional neural networks

3D Spatial Pyramid Dilated Network for Pulmonary Nodule Classification.

Spatial Pyramid Dilated Network for Pulmonary Nodule Malignancy Classification.

A Deep-Learning Based Automatic Pulmonary Nodule Detection System

A Novel Pulmonary Nodule Detection Model Based on Multi-Step Cascaded Networks

MSCS-DeepLN: Evaluating Lung Nodule Malignancy Using Multi-Scale Cost-Sensitive Neural Networks

A Novel Self-Adaptive Convolutional Neural Network Model Using Spatial Pyramid Pooling for 3D Lung Nodule Computer-Aided Diagnosis

Pulmonary Nodule Detection Using 3-D Residual U-Net Oriented Context-Guided Attention and Multi-Branch Classification Network

An attention-based deep learning network for lung nodule malignancy discrimination

A Novel Deep Learning Model Based on Multi-Scale and Multi-View for Detection of Pulmonary Nodules

Identification of Pulmonary Nodules Via CT Images with Hierarchical Fully Convolutional Networks.

3D Multi-Branch Encoder-Decoder Networks with Attentional Feature Fusion for Pulmonary Nodule Detection in CT Scans.

Attention-guided deep neural network with a multichannel architecture for lung nodule classification

Lung Nodule Diagnosis on 3D Computed Tomography Images Using Deep Convolutional Neural Networks

Lung Nodule Classification of CT Images Based on the Deep Learning Algorithms

3Dagnet: 3D Deep Attention and Global Search Network for Pulmonary Nodule Detection

Automatic Lung Nodule Detection Using a 3D Deep Convolutional Neural Network Combined with a Multi-Scale Prediction Strategy in Chest CTs