Abstract:Transfer learning enables to re-use knowledge learned on a source task to help learning a target task. A simple form of transfer learning is common in current state-of-the-art computer vision models, i.e. pre-training a model for image classification on the ILSVRC dataset, and then fine-tune on any target task. However, previous systematic studies of transfer learning have been limited and the circumstances in which it is expected to work are not fully understood. In this paper we carry out an extensive experimental exploration of transfer learning across vastly different image domains (consumer photos, autonomous driving, aerial imagery, underwater, indoor scenes, synthetic, close-ups) and task types (semantic segmentation, object detection, depth estimation, keypoint detection). Importantly, these are all complex, structured output tasks types relevant to modern computer vision applications. In total we carry out over 2000 transfer learning experiments, including many where the source and target come from different image domains, task types, or both. We systematically analyze these experiments to understand the impact of image domain, task type, and dataset size on transfer learning performance. Our study leads to several insights and concrete recommendations: (1) for most tasks there exists a source which significantly outperforms ILSVRC'12 pre-training; (2) the image domain is the most important factor for achieving positive transfer; (3) the source dataset should \emph{include} the image domain of the target dataset to achieve best results; (4) at the same time, we observe only small negative effects when the image domain of the source task is much broader than that of the target; (5) transfer across task types can be beneficial, but its success is heavily dependent on both the source and target task types.

Ontology-Driven Cross-Domain Transfer Learning

Transfer Learning and Deep Domain Adaptation

Graph Enabled Cross-Domain Knowledge Transfer

Improving the Generalization Ability of Deep Neural Networks for Cross-Domain Visual Recognition

Factors of Influence for Transfer Learning across Diverse Appearance Domains and Task Types

Bridging Domains Using World Wide Knowledge for Transfer Learning

Human-centric Transfer Learning Explanation Via Knowledge Graph [extended Abstract]

Cross-domain and Cross-dimension Learning for Image-to-Graph Transformers

Softly Associative Transfer Learning for Cross-Domain Classification

Multilevel Knowledge Transfer for Cross-Domain Object Detection

Knowledge-based Transfer Learning Explanation

Multi-domain few-shot image recognition with knowledge transfer

Cross-Domain Kernel Induction for Transfer Learning

Open-world Domain Adaptation and Generalization

Cross-domain NER under a Divide-and-Transfer Paradigm

Domain Adaptation via Transfer Component Analysis

Cross-Modality Bridging and Knowledge Transferring for Image Understanding

Improving Generalization of Transfer Learning Across Domains Using Spatio-Temporal Features in Autonomous Driving

Cross-Transfer Learning for Enhancing Object Detection in Remote Sensing Images

Cross-domain attribute representation based on convolutional neural network

Deep Transfer Neural Network Using Hybrid Representations of Domain Discrepancy