Abstract:Abstract Background Transfer learning (TL) with convolutional neural networks aims to improve performances on a new task by leveraging the knowledge of similar tasks learned in advance. It has made a major contribution to medical image analysis as it overcomes the data scarcity problem as well as it saves time and hardware resources. However, transfer learning has been arbitrarily configured in the majority of studies. This review paper attempts to provide guidance for selecting a model and TL approaches for the medical image classification task. Methods 425 peer-reviewed articles were retrieved from two databases, PubMed and Web of Science, published in English, up until December 31, 2020. Articles were assessed by two independent reviewers, with the aid of a third reviewer in the case of discrepancies. We followed the PRISMA guidelines for the paper selection and 121 studies were regarded as eligible for the scope of this review. We investigated articles focused on selecting backbone models and TL approaches including feature extractor, feature extractor hybrid, fine-tuning and fine-tuning from scratch. Results The majority of studies (n = 57) empirically evaluated multiple models followed by deep models (n = 33) and shallow (n = 24) models. Inception, one of the deep models, was the most employed in literature (n = 26). With respect to the TL, the majority of studies (n = 46) empirically benchmarked multiple approaches to identify the optimal configuration. The rest of the studies applied only a single approach for which feature extractor (n = 38) and fine-tuning from scratch (n = 27) were the two most favored approaches. Only a few studies applied feature extractor hybrid (n = 7) and fine-tuning (n = 3) with pretrained models. Conclusion The investigated studies demonstrated the efficacy of transfer learning despite the data scarcity. We encourage data scientists and practitioners to use deep models (e.g. ResNet or Inception) as feature extractors, which can save computational costs and time without degrading the predictive power.

An Abridged Review of Transfer Learning Technology

Transfer Learning and Deep Domain Adaptation

Transfer Learning Applied to Computer Vision Problems: Survey on Current Progress, Limitations, and Opportunities

Transferability in Deep Learning: A Survey

A Review of Deep Learning on Medical Image Analysis

A Comprehensive Survey on Transfer Learning

Factors of Influence for Transfer Learning across Diverse Appearance Domains and Task Types

Transfer learning in robotics: An upcoming breakthrough? A review of promises and challenges

A Visual Analytical Approach for Transfer Learning in Classification.

Deep Learning for Computer Vision: A Brief Review

Transfer learning for medical image classification: a literature review

Transfer Learning and Its Application Research

Transfer learning for medical images analyses: A survey

Transfer learning: a friendly introduction

Knowledge Transfer in Vision Recognition

The Utility of Feature Reuse: Transfer Learning in Data-Starved Regimes

Transfusion: Understanding Transfer Learning for Medical Imaging

A Survey on Deep Transfer Learning.

A Comprehensive Review of Knowledge Distillation in Computer Vision

A Review of Deep Transfer Learning and Recent Advancements

Transfer Learning in Deep Reinforcement Learning: A Survey