Abstract:Humans are capable of accumulating knowledge by sequentially learning different tasks, while neural networks fail to achieve this due to catastrophic forgetting problems. Most current incremental learning methods focus more on tackling catastrophic forgetting for traditional classification networks. Notably, however, embedding networks that are basic architectures for many metric learning applications also suffer from this problem. Moreover, the most significant difficulty for continual embedding networks is that the relationships between the latent features and prototypes of previous tasks will be destroyed once new tasks have been learned. Accordingly, we propose a novel incremental method for embedding networks, called the disentangled representation translation (DRT) method, to obtain the discriminative class-disentangled features without reusing any samples of previous tasks and while avoiding the perturbation of task-related information. Next, a mask-guided module is specifically explored to adaptively change or retain the valuable information of latent features. This module enables us to effectively preserve the discriminative yet representative features in the disentangled translation process. In addition, DRT can easily be equipped with a regularization item of incremental learning to further improve performance. We conduct extensive experiments on four popular datasets; as the experimental results clearly demonstrate, our method can effectively alleviate the catastrophic forgetting problem for embedding networks.

Approximate Error Estimation based Incremental Word Representation Learning

Incremental Learning Algorithm Based on Support Vector Machine

An Incremental Learning Algorithm Based on Support Vector Domain Classifier

Entropy-Based Vocabulary Substitution for Incremental Learning in Multilingual Neural Machine Translation

Incremental class learning using variational autoencoders with similarity learning

Approximating Pareto Optimal Set by An Incremental Learning Model

Contrastive-based Bi-update Class Incremental Model and Its Application in Fault Diagnosis

Stochastic Backpropagation and Approximate Inference in Deep Generative Models

An Online Incremental Language Model Adaptation Method

On Learning Errors of Structured Prediction with Approximate Inference

Incremental Embedding Learning with Disentangled Representation Translation.

Hybrid Incremental Learning Algorithms for Bayesian Network Structures

Incremental Updating Multigranulation Approximations with Matrix Representation Under Coarsening Decision Attribute

Learning Word Representation Considering Proximity and Ambiguity

Incremental learning Bayesian network structures efficiently

INCREMENTAL LEARNING FOR END-TO-END AUTOMATIC SPEECH RECOGNITION

Efficient Incremental Belief Updates Using Weighted Virtual Observations

An Incremental Bayesian Network Structure Learning Algorithm Based on Local Updating Strategy

Deep Neural Network-Based Algorithm Approximation Via Multivariate Polynomial Regression

Estimator Vectors: OOV Word Embeddings based on Subword and Context Clue Estimates