Abstract:Transfer learning enables to re-use knowledge learned on a source task to help learning a target task. A simple form of transfer learning is common in current state-of-the-art computer vision models, i.e. pre-training a model for image classification on the ILSVRC dataset, and then fine-tune on any target task. However, previous systematic studies of transfer learning have been limited and the circumstances in which it is expected to work are not fully understood. In this paper we carry out an extensive experimental exploration of transfer learning across vastly different image domains (consumer photos, autonomous driving, aerial imagery, underwater, indoor scenes, synthetic, close-ups) and task types (semantic segmentation, object detection, depth estimation, keypoint detection). Importantly, these are all complex, structured output tasks types relevant to modern computer vision applications. In total we carry out over 2000 transfer learning experiments, including many where the source and target come from different image domains, task types, or both. We systematically analyze these experiments to understand the impact of image domain, task type, and dataset size on transfer learning performance. Our study leads to several insights and concrete recommendations: (1) for most tasks there exists a source which significantly outperforms ILSVRC'12 pre-training; (2) the image domain is the most important factor for achieving positive transfer; (3) the source dataset should \emph{include} the image domain of the target dataset to achieve best results; (4) at the same time, we observe only small negative effects when the image domain of the source task is much broader than that of the target; (5) transfer across task types can be beneficial, but its success is heavily dependent on both the source and target task types.

Knowledge Transfer in Vision Recognition

A Visual Analytical Approach for Transfer Learning in Classification.

Transferability in Deep Learning: A Survey

Human-centric Transfer Learning Explanation Via Knowledge Graph [extended Abstract]

Continual Learning with Knowledge Transfer for Sentiment Classification

K-LITE: Learning Transferable Visual Models with External Knowledge

An Abridged Review of Transfer Learning Technology

Data-Free Knowledge Transfer: A Survey

A Comprehensive Survey on Transfer Learning

Transferring Vision-Language Models for Visual Recognition: A Classifier Perspective

Factors of Influence for Transfer Learning across Diverse Appearance Domains and Task Types

Transfer Learning and Its Application Research

A Survey on Heterogeneous Transfer Learning

CrowdTransfer: Enabling Crowd Knowledge Transfer in AIoT Community

Exploring Simple and Transferable Recognition-Aware Image Processing

The Contribution of Knowledge in Visiolinguistic Learning: A Survey on Tasks and Challenges

Transfer: Cross Modality Knowledge Transfer using Adversarial Networks -- A Study on Gesture Recognition

Knowledge Transfer and Domain Adaptation for Fine-Grained Remote Sensing Image Segmentation

Transferring Knowledge from Vision to Language: How to Achieve it and how to Measure it?

Dynamic Knowledge Distillation with Cross-Modality Knowledge Transfer

Knowledge Transfer from Vision Foundation Models for Efficient Training of Small Task-specific Models