Abstract:Model reuse attempts to construct a model by utilizing existing available models, mostly trained for other tasks, rather than building a model from scratch. It is helpful to reduce the time cost, data amount, and expertise required. Deep learning has achieved great success in various tasks involving images, voices and videos. There are several studies have the sense of model reuse, by trying to reuse pre-trained deep networks architectures or deep model features to train a new deep model. They, however, neglect the fact that there are many other fixed models or features available. In this paper, we propose a more thorough model reuse scheme, FMR (Fixed Model Reuse). FMR utilizes the learning power of deep models to implicitly grab the useful discriminative information from fixed model/features that have been widely used in general tasks. We firstly arrange the convolution layers of a deep network and the provided fixed model/features in parallel, fully connecting to the output layer nodes. Then, the dependencies between the output layer nodes and the fixed model/features are knockdown such that only the raw feature inputs are needed when the model is being used for testing, though the helpful information in the fixed model/features have already been incorporated into the model. On one hand, by the FMR scheme, the required amount of training data can be significantly reduced because of the reuse of fixed model/features. On the other hand, the fixed model/features are not explicitly used in testing, and thus, the scheme can be quite useful in applications where the fixed model/features are protected by patents or commercial secrets. Experiments on five real-world datasets validate the effectiveness of FMR compared with state-of-the-art deep methods.

Model Reuse with Domain Knowledge

Towards Robust Model Reuse in the Presence of Latent Domains

Nonlinear Multi-Model Reuse.

Deep Learning for Fixed Model Reuse.

ZhiJian: A Unifying and Rapidly Deployable Toolbox for Pre-trained Model Reuse

Model Reuse with Reduced Kernel Mean Embedding Specification

Reusing Deep Neural Network Models through Model Re-engineering

Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains.

Handling Concept Drift via Model Reuse

An Empirical Study of Pre-Trained Model Reuse in the Hugging Face Deep Learning Model Registry

Learnware: Small Models Do Big

Modal Consistency Based Pre-Trained Multi-Model Reuse.

Efficient Model Store and Reuse in an OLML Database System

Model Uncertainty-Aware Knowledge Amalgamation for Pre-Trained Language Models

ModelFoundry: A Tool for DNN Modularization and On-Demand Model Reuse Inspired by the Wisdom of Software Engineering

Post deployment recycling of machine learning models

Toward Effective Mechanical Design Reuse: CAD Model Retrieval Based on General and Partial Shapes

Boosting-Based Reliable Model Reuse.

Design Reusability Assessment For Effective Cad Model Retrieval And Reuse

The reusability prior: comparing deep learning models without training

Drop Redundant, Shrink Irrelevant: Selective Knowledge Injection for Language Pretraining