Abstract:Abstract Catastrophic forgetting in neural networks is a common problem, in which neural networks lose information from previous tasks after training on new tasks. Although adopting a regularization method that preferentially retains the parameters important to the previous task to avoid catastrophic forgetting has a positive effect; existing regularization methods cause the gradient to be near zero because the loss is at the local minimum. To solve this problem, we propose a new continuous learning method with Bayesian parameter updating and weight memory (CL-BPUWM). First, a parameter updating method based on the Bayes criterion is proposed to allow the neural network to gradually obtain new knowledge. The diagonal of the Fisher information matrix is then introduced to significantly minimize computation and increase parameter updating efficiency. Second, we suggest calculating the importance weight by observing how changes in each network parameter affect the model prediction output. In the process of model parameter updating, the Fisher information matrix and the sensitivity of the network are used as the quadratic penalty terms of the loss function. Finally, we apply dropout regularization to reduce model overfitting during training and to improve model generalizability. CL-BPUWM performs very well in continuous learning for classification tasks on CIFAR-100 dataset, CIFAR-10 dataset, and MNIST dataset. On CIFAR-100 dataset, it is 0.8%, 1.03% and 0.75% higher than the best performing regularization method (EWC) in three task partitions. On CIFAR-10 dataset, it is 2.25% higher than the regularization method (EWC) and 0.7% higher than the scaled method (GR). It is 0.66% higher than the regularization method (EWC) on the MNIST dataset. When the CL-BPUWM method was combined with the brain-inspired replay model under the CIFAR-100 and CIFAR-10 datasets, the classification accuracy was 2.35% and 5.38% higher than that of the baseline method, BI-R + SI.

Regularization Continual Learning Based on Bayesian Uncertainty Modeling for P300 Brain-Computer Interface

Bayesian Uncertainty Modeling for P300-Based Brain-Computer Interface

Subject-Independent Wearable P300 Brain-Computer Interface Based on Convolutional Neural Network and Metric Learning

A P300-Detection Method Based on Logistic Regression and a Convolutional Neural Network

Learning Invariant Patterns Based on a Convolutional Neural Network and Big Electroencephalography Data for Subject-Independent P300 Brain-Computer Interfaces

Eliminating or Shortening the Calibration for a P300 Brain–Computer Interface Based on a Convolutional Neural Network and Big Electroencephalography Data: An Online Study

Inter-participant transfer learning with attention based domain adversarial training for P300 detection

Improving P300 Speller performance by means of optimization and machine learning

Bayesian Signal Matching for Transfer Learning in ERP-Based Brain Computer Interface

Weighted Common Spatial Pattern Based Adaptation Regularization for Multi-Source Eeg Time Series

The Study of Generic Model Set for Reducing Calibration Time in P300-Based Brain–Computer Interface

A Regional Smoothing Block Sparse Bayesian Learning Method with Temporal Correlation for Channel Selection in P300 Speller.

Towards Effective Deep Neural Network Approach for Multi-Trial P300-based Character Recognition in Brain-Computer Interfaces

Leveraging Deep Learning Techniques to Improve P300-Based Brain Computer Interfaces

CL-BPUWM: continuous learning with Bayesian parameter updating and weight memory

End-to-end P300 BCI using Bayesian accumulation of Riemannian probabilities

Fuzzy temporal convolutional neural networks in P300-based Brain-computer interface for smart home interaction

Performance Enhancement of P300 Detection by Multiscale-CNN

A cross-subject decoding algorithm for patients with disorder of consciousness based on P300 brain computer interface

Optimization of transfer learning based on source sample selection in Euclidean space for P300-based brain-computer interfaces

A P300-Based Bci Classification Algorithm Using Median Filtering and Bayesian Feature Extraction