Abstract:ObjectivesThe interpretability of convolutional neural networks (CNNs) for classifying subsolid nodules (SSNs) is insufficient for clinicians. Our purpose was to develop CNN models to classify SSNs on CT images and to investigate image features associated with the CNN classification.MethodsCT images containing SSNs with a diameter of ≤ 3 cm were retrospectively collected. We trained and validated CNNs by a 5-fold cross-validation method for classifying SSNs into three categories (benign and preinvasive lesions [PL], minimally invasive adenocarcinoma [MIA], and invasive adenocarcinoma [IA]) that were histologically confirmed or followed up for 6.4 years. The mechanism of CNNs on human-recognizable CT image features was investigated and visualized by gradient-weighted class activation map (Grad-CAM), separated activation channels and areas, and DeepDream algorithm.ResultsThe accuracy was 93% for classifying 586 SSNs from 569 patients into three categories (346 benign and PL, 144 MIA, and 96 IA in 5-fold cross-validation). The Grad-CAM successfully located the entire region of image features that determined the final classification. Activated areas in the benign and PL group were primarily smooth margins (p < 0.001) and ground-glass components (p = 0.033), whereas in the IA group, the activated areas were mainly part-solid (p < 0.001) and solid components (p < 0.001), lobulated shapes (p < 0.001), and air bronchograms (p < 0.001). However, the activated areas for MIA were variable. The DeepDream algorithm showed the image features in a human-recognizable pattern that the CNN learned from a training dataset.ConclusionThis study provides medical evidence to interpret the mechanism of CNNs that helps support the clinical application of artificial intelligence.Key Points• CNN achieved high accuracy (93%) in classifying subsolid nodules on CT images into three categories: benign and preinvasive lesions, MIA, and IA.• The gradient-weighted class activation map (Grad-CAM) located the entire region of image features that determined the final classification, and the visualization of the separated activated areas was consistent with radiologists’ expertise for diagnosing subsolid nodules.• DeepDream showed the image features that CNN learned from a training dataset in a human-recognizable pattern.

Shape and Margin-Aware Lung Nodule Classification in Low-dose CT Images via Soft Activation Mapping

Soft Activation Mapping of Lung Nodules in Low-Dose CT images.

Classification of Lung Nodules Based on Deep Residual Networks and Migration Learning

A Hybrid CNN Feature Model for Pulmonary Nodule Malignancy Risk Differentiation.

A Deep 3D Residual CNN for False-Positive Reduction in Pulmonary Nodule Detection

Optimized convolutional neural network for automatic lung nodule detection with a new active contour segmentation

Joint Learning for Pulmonary Nodule Segmentation, Attributes and Malignancy Prediction

LDCT image biomarkers that matter most for the deep learning classification of indeterminate pulmonary nodules

Automated Pulmonary Nodule Detection in CT Images Using Deep Convolutional Neural Networks.

Detection-Guided Deep Learning-Based Model with Spatial Regularization for Lung Nodule Segmentation

Hyperparameter optimization and development of an advanced CNN-based technique for lung nodule assessment

Lung Nodule Segmentation Combining Sobel Operator and Mask R-CNN

Attention-guided deep neural network with a multichannel architecture for lung nodule classification

Central focused convolutional neural networks: Developing a data-driven model for lung nodule segmentation

3D Multi-View Squeeze-and-excitation Convolutional Neural Network for Lung Nodule Classification

Multi-kernel driven 3D convolutional neural network for automated detection of lung nodules in chest CT scans

An improved 3-D attention CNN with hybrid loss and feature fusion for pulmonary nodule classification

Human-recognizable CT image features of subsolid lung nodules associated with diagnosis and classification by convolutional neural networks

Discriminative Localization in CNNs for Weakly-Supervised Segmentation of Pulmonary Nodules

Fusing Texture, Shape and Deep Model-Learned Information at Decision Level for Automated Classification of Lung Nodules on Chest CT.

Multiscale lung nodule segmentation based on 3D coordinate attention and edge enhancement