Abstract:Motivation In most biological data sets, the amount of data is regularly growing and the number of classes is continuously increasing. To deal with the new data from the new classes, one approach is to train a classification model, e.g., a deep learning model, from scratch based on both old and new data. This approach is highly computationally costly and the extracted features are likely very different from the ones extracted by the model trained on the old data alone, which leads to poor model robustness. Another approach is to fine tune the trained model from the old data on the new data. However, this approach often does not have the ability to learn new knowledge without forgetting the previously learned knowledge, which is known as the catastrophic forgetting problem. To our knowledge, this problem has not been studied in the field of bioinformatics despite its existence in many bioinformatic problems. Results Here we propose a novel method, SupportNet, to solve the catastrophic forgetting problem efficiently and effectively. SupportNet combines the strength of deep learning and support vector machine (SVM), where SVM is used to identify the support data from the old data, which are fed to the deep learning model together with the new data for further training so that the model can review the essential information of the old data when learning the new information. Two powerful consolidation regularizers are applied to ensure the robustness of the learned model. Comprehensive experiments on various tasks, including enzyme function prediction, subcellular structure classification and breast tumor classification, show that SupportNet drastically outperforms the state-of-the-art incremental learning methods and reaches similar performance as the deep learning model trained from scratch on both old and new data. Availability Our program is accessible at: <https://github.com/lykaust15/SupportNet>.

SupportNet: solving catastrophic forgetting in class incremental learning with support data.

SupportNet: a novel incremental learning framework through deep learning and support data

An Efficient Strategy for Catastrophic Forgetting Reduction in Incremental Learning

Mitigating Catastrophic Forgetting in Cross-Domain Fault Diagnosis: An Unsupervised Class Incremental Learning Network Approach

SynapNet: A Complementary Learning System Inspired Algorithm With Real-Time Application in Multimodal Perception

Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting

Class-Incremental Learning via Knowledge Amalgamation

Maintaining Discrimination and Fairness in Class Incremental Learning

EXACFS -- A CIL Method to mitigate Catastrophic Forgetting

Exemplar-Supported Representation for Effective Class-Incremental Learning

Large Scale Incremental Learning

Overcoming catastrophic forgetting in neural networks

Adaptive knowledge transfer for class incremental learning

Overcoming Catastrophic Forgetting with Unlabeled Data in the Wild

A Robust and Anti-Forgettiable Model for Class-Incremental Learning

Class incremental learning with self-supervised pre-training and prototype learning

Model Behavior Preserving for Class-Incremental Learning

Class-Incremental Learning Based on Big Dataset Pre-Trained Models

Catastrophic forgetting: still a problem for DNNs

Complementary Learning Subnetworks for Parameter-Efficient Class-Incremental Learning

Measuring Catastrophic Forgetting in Neural Networks