Abstract:Purpose: To perform an in-depth evaluation of current state of the art techniques in training neural networks to identify appropriate approaches in small datasets. Method: In total, 112,120 frontal-view X-ray images from the NIH ChestXray14 dataset were used in our analysis. Two tasks were studied: unbalanced multi-label classification of 14 diseases, and binary classification of pneumonia vs non-pneumonia. All datasets were randomly split into training, validation, and testing (70%, 10%, and 20%). Two popular convolution neural networks (CNNs), DensNet121 and ResNet50, were trained using PyTorch. We performed several experiments to test: (a) whether transfer learning using pretrained networks on ImageNet are of value to medical imaging/physics tasks (e.g., predicting toxicity from radiographic images after training on images from the internet), (b) whether using pretrained networks trained on problems that are similar to the target task helps transfer learning (e.g., using X-ray pretrained networks for X-ray target tasks), (c) whether freeze deep layers or change all weights provides an optimal transfer learning strategy, (d) the best strategy for the learning rate policy, and (e) what quantity of data is needed in order to appropriately deploy these various strategies (N = 50 to N = 77 880). Results: In the multi-label problem, DensNet121 needed at least 1600 patients to be comparable to, and 10 000 to outperform, radiomics-based logistic regression. In classifying pneumonia vs non-pneumonia, both CNN and radiomics-based methods performed poorly when N < 2000. For small datasets ( < 2000), however, a significant boost in performance (>15% increase on AUC) comes from a good selection of the transfer learning dataset, dropout, cycling learning rate, and freezing and unfreezing of deep layers as training progresses. In contrast, if sufficient data are available (>35 000), little or no tweaking is needed to obtain impressive performance. While transfer learning using X-ray images from other anatomical sites improves performance, we also observed a similar boost by using pretrained networks from ImageNet. Having source images from the same anatomical site, however, outperforms every other methodology, by up to 15%. In this case, DL models can be trained with as little as N = 50. Conclusions: While training DL models in small datasets (N < 2000) is challenging, no tweaking is necessary for bigger datasets (N > 35 000). Using transfer learning with images from the same anatomical site can yield remarkable performance in new tasks with as few as N = 50. Surprisingly, we did not find any advantage for using images from other anatomical sites over networks that have been trained using ImageNet. This indicates that features learned may not be as general as currently believed, and performance decays rapidly even by just changing the anatomical site of the images.

Migrating Knowledge between Physical Scenarios based on Artificial Neural Networks

Network-scale traffic prediction via knowledge transfer and regional MFD analysis

Short-term Traffic Prediction with Deep Neural Networks and Adaptive Transfer Learning

Evaluating Knowledge Transfer in Neural Network for Medical Images

Targeted transfer learning to improve performance in small medical physics datasets

Transfer learning for metamaterial design and simulation

Knowledge Transfer between Datasets for Learning-based Tissue Microstructure Estimation

Deep Transfer Learning for Intelligent Cellular Traffic Prediction Based on Cross-Domain Big Data

Transfer Knowledge Between Sub-Regions For Traffic Prediction Using Deep Learning Method

Learning What and Where to Transfer

Improvement of Applicability in Student Performance Prediction Based on Transfer Learning

Watch and learn -- a generalized approach for transferrable learning in deep neural networks via physical principles

A novel neural network architecture and cross-model transfer learning for multi-task autonomous driving

Addressing Data Scarcity in Optical Matrix Multiplier Modeling Using Transfer Learning

Transfer learning based on atomic feature extraction for the prediction of experimental ¹³C chemical shifts

Robust Knowledge Transfer Via Hybrid Forward on the Teacher-Student Model

A 3D ray traced biological neural network learning model

NodeTrans: A Graph Transfer Learning Approach for Traffic Prediction

Trans-Net: A transferable pretrained neural networks based on temporal domain decomposition for solving partial differential equations

Modeling unobserved geothermal structures using a physics-informed neural network with transfer learning of prior knowledge

Network-Agnostic Knowledge Transfer for Medical Image Segmentation