Transfer Learning of Fuzzy Spatio-Temporal Rules in the NeuCube Brain-Inspired Spiking Neural Network: A Case Study on EEG Spatio-temporal Data

Nikola Kasabov,Yongyao Tan Tan,Maryam Doborjeh,Enmei Tu,Jie Yang

DOI: https://doi.org/10.36227/techrxiv.21781103

2023-01-01

Abstract:<p>EEG data collected from several subjects when perfoming complex spatio-temporal tasks </p>

What problem does this paper attempt to address?

Transfer Learning of Fuzzy Spatio-Temporal Rules in a Brain-Inspired Spiking Neural Network Architecture: A Case Study on Spatio-Temporal Brain Data

Nikola K. Kasabov,Yongyao Tan,Maryam Doborjeh,Enmei Tu,Jie Yang,Wilson Goh,Jimmy Lee

DOI: https://doi.org/10.1109/TFUZZ.2023.3292802

IF: 12.253

2023-01-01

IEEE Transactions on Fuzzy Systems

Abstract:The article demonstrates for the first time that a brain-inspired spiking neural network (SNN) architecture can be used not only to learn spatio-temporal data, but also to extract fuzzy spatio-temporal rules from such data and to update these rules incrementally in a transfer learning mode. We propose a method, where a SNN model learns incrementally new time-space data related to new classes/tasks/categories, always utilizing some previously learned knowledge, and presents the evolved knowledge as fuzzy spatio-temporal rules. Similarly, to how the brain manifests transfer learning, these SNN models do not need to be restricted in number of layers and neurons in each layer as they adopt self-organizing learning principles. The continuously evolved fuzzy rules from spatio-temporal data are interpretable for a better understanding of the processes that generate the data. The proposed method is based on a brain-inspired SNN architecture NeuCube, which is structured according to a brain three-dimensional structural template. It is illustrated on tasks of incremental and transfer learning and knowledge transfer using spatio-temporal data measuring brain activity, when subjects are performing tasks in space and time. The method is a general one and opens the field to create new types of adaptable and explainable spatio-temporal learning systems across domain areas.
Improving NeuCube Spiking Neural Network for EEG-based Pattern Recognition Using Transfer Learning

Xuanyu Wu,Yixiong Feng,Shanhe Lou,Hao Zheng,Bingtao Hu,Zhaoxi Hong,Jianrong Tan

DOI: https://doi.org/10.1016/j.neucom.2023.01.087

IF: 6

2023-01-01

Neurocomputing

Abstract:Electroencephalogram (EEG) data are produced in quantity for measuring brain activity in response to external stimuli. With the rapid development of brain-inspired intelligence, spiking neural network (SNN) possesses the potential to handle EEG data by using spiking activity transmitted among spatially located synapses and neurons. As an original and unifying SNN architecture, NeuCube, is developed to model, recognize and understand EEG data. However, the NeuCube still faces some challenges for EEG-based pattern recognition, such as few labeled data and changes of data probability distribution. Hence, this paper proposes a novel method to improve the performance of the NeuCube for EEG-based pattern recognition by transfer learning. In the first place, the covariance matrix alignment of EEG data is implemented for every subject in the Euclidean space, which reduces the probability distribution discrepancy of EEG data between different subjects. Different estimation methods for reference covariance matrix are tested and the optimal one is selected for different subjects. Secondly, spatio-temporal features of EEG data are extracted based on the NeuCube reservoir. Since hyper-parameters of the NeuCube reservoir have a great impact on its spatio-temporal representation, an improved cuckoo search algorithm is proposed to discover the optimal hyper-parameters for obtaining the optimal spatio-temporal features. Last but not least, a weighted transfer support vector machine is proposed to improve the original output classifier of the NeuCube in order to make the model adaptive to the cross-domain variability of EEG data. The proposed method is tested on open dataset 2a from BCI competition IV 2008 and achieves good spatio-temporal pattern recognition results. Furthermore, the neuron connectivity and activation level associated with the process of mental tasks are illustrated.
Mapping Temporal Variables into the NeuCube for Improved Pattern Recognition, Predictive Modeling, and Understanding of Stream Data

Enmei Tu,Nikola Kasabov,Jie Yang

DOI: https://doi.org/10.1109/tnnls.2016.2536742

IF: 14.255

2017-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:This paper proposes a new method for an optimized mapping of temporal variables, describing a temporal stream data, into the recently proposed NeuCube spiking neural network (SNN) architecture. This optimized mapping extends the use of the NeuCube, which was initially designed for spatiotemporal brain data, to work on arbitrary stream data and to achieve a better accuracy of temporal pattern recognition, a better and earlier event prediction, and a better understanding of complex temporal stream data through visualization of the NeuCube connectivity. The effect of the new mapping is demonstrated on three benchmark problems. The first one is the early prediction of patient sleep stage event from temporal physiological data. The second one is the pattern recognition of dynamic temporal patterns of traffic in the Bay Area of California and the last one is the Challenge 2012 contest data set. In all the cases, the use of the proposed mapping leads to an improved accuracy of pattern recognition and event prediction and a better understanding of the data when compared with traditional machine learning techniques or SNN reservoirs with an arbitrary mapping of the variables.
A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

Qiang Yu,Huajin Tang,Kay Chen Tan,Haoyong Yu

DOI: https://doi.org/10.1016/j.neucom.2013.06.052

IF: 6

2014-01-01

Neurocomputing

Abstract:Neural coding and learning are important components in cognitive memory system, by processing the sensory inputs and distinguishing different patterns to allow for higher level brain functions such as memory storage and retrieval. Benefitting from biological relevance, this paper presents a spiking neural network of leaky integrate-and-fire (LIF) neurons for pattern recognition. A biologically plausible supervised synaptic learning rule is used so that neurons can efficiently make a decision. The whole system contains encoding, learning and readout. Utilizing the temporal coding and learning, networks of spiking neurons can effectively and efficiently perform various classification tasks. It can classify complex patterns of activities stored in a vector, as well as the real-world stimuli. Our approach is also benchmarked on the nonlinearly separable Iris dataset. The proposed approach achieves a good generalization, with a classification accuracy of 99.63% for training and 92.55% for testing. In addition, the trained networks demonstrate that the temporal coding is a viable means for fast neural information processing.
NeuCube(ST) for spatio-temporal data predictive modelling with a case study on ecological data

Enmei Tu,Nikola K. Kasabov,Muhaini Othman,Yuxiao Li,Susan P. Worner,Jie Yang,Zhenghong Jia

DOI: https://doi.org/10.1109/IJCNN.2014.6889717

IF: 7.8

2014-01-01

Neural Networks

Abstract:Early event prediction challenges most of existing modeling methods especially when dealing with complex spatio-temporal data. In this paper we propose a new method for predictive data modelling based on a new development of the recently proposed NeuCube spiking neural network architecture, called here NeuCube(ST). The NeuCube uses a Spiking Neural Network reservoir (SNNr) and dynamic evolving Spiking Neuron Network (deSNN) classifier. NeuCube(ST) is an integrated environment including data conversion into spike trains, input variable mapping, unsupervised learning in the SNNr, supervised classification learning, activity visualization and network structure analysis. A case study on a real world ecological data set is presented to demonstrate the validity of the proposed method.
NeuCube: A spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data

Nikola K. Kasabov

DOI: https://doi.org/10.1016/j.neunet.2014.01.006

IF: 7.8

2014-04-01

Neural Networks

Abstract:The brain functions as a spatio-temporal information processing machine. Spatio- and spectro-temporal brain data (STBD) are the most commonly collected data for measuring brain response to external stimuli. An enormous amount of such data has been already collected, including brain structural and functional data under different conditions, molecular and genetic data, in an attempt to make a progress in medicine, health, cognitive science, engineering, education, neuro-economics, Brain-Computer Interfaces (BCI), and games. Yet, there is no unifying computational framework to deal with all these types of data in order to better understand this data and the processes that generated it. Standard machine learning techniques only partially succeeded and they were not designed in the first instance to deal with such complex data. Therefore, there is a need for a new paradigm to deal with STBD. This paper reviews some methods of spiking neural networks (SNN) and argues that SNN are suitable for the creation of a unifying computational framework for learning and understanding of various STBD, such as EEG, fMRI, genetic, DTI, MEG, and NIRS, in their integration and interaction. One of the reasons is that SNN use the same computational principle that generates STBD, namely spiking information processing. This paper introduces a new SNN architecture, called NeuCube, for the creation of concrete models to map, learn and understand STBD. A NeuCube model is based on a 3D evolving SNN that is an approximate map of structural and functional areas of interest of the brain related to the modeling STBD. Gene information is included optionally in the form of gene regulatory networks (GRN) if this is relevant to the problem and the data. A NeuCube model learns from STBD and creates connections between clusters of neurons that manifest chains (trajectories) of neuronal activity. Once learning is applied, a NeuCube model can reproduce these trajectories, even if only part of the input STBD or the stimuli data is presented, thus acting as an associative memory. The NeuCube framework can be used not only to discover functional pathways from data, but also as a predictive system of brain activities, to predict and possibly, prevent certain events. Analysis of the internal structure of a model after training can reveal important spatio-temporal relationships 'hidden' in the data. NeuCube will allow the integration in one model of various brain data, information and knowledge, related to a single subject (personalized modeling) or to a population of subjects. The use of NeuCube for classification of STBD is illustrated in a case study problem of EEG data. NeuCube models result in a better accuracy of STBD classification than standard machine learning techniques. They are robust to noise (so typical in brain data) and facilitate a better interpretation of the results and understanding of the STBD and the brain conditions under which data was collected. Future directions for the use of SNN for STBD are discussed.

computer science, artificial intelligence,neurosciences
Deep Learning of Explainable EEG Patterns as Dynamic Spatiotemporal Clusters and Rules in a Brain-Inspired Spiking Neural Network

Maryam Doborjeh,Zohreh Doborjeh,Nikola Kasabov,Molood Barati,Grace Y Wang

DOI: https://doi.org/10.3390/s21144900

2021-07-19

Abstract:The paper proposes a new method for deep learning and knowledge discovery in a brain-inspired Spiking Neural Networks (SNN) architecture that enhances the model's explainability while learning from streaming spatiotemporal brain data (STBD) in an incremental and on-line mode of operation. This led to the extraction of spatiotemporal rules from SNN models that explain why a certain decision (output prediction) was made by the model. During the learning process, the SNN created dynamic neural clusters, captured as polygons, which evolved in time and continuously changed their size and shape. The dynamic patterns of the clusters were quantitatively analyzed to identify the important STBD features that correspond to the most activated brain regions. We studied the trend of dynamically created clusters and their spike-driven events that occur together in specific space and time. The research contributes to: (1) enhanced interpretability of SNN learning behavior through dynamic neural clustering; (2) feature selection and enhanced accuracy of classification; (3) spatiotemporal rules to support model explainability; and (4) a better understanding of the dynamics in STBD in terms of feature interaction. The clustering method was applied to a case study of Electroencephalogram (EEG) data, recorded from a healthy control group (n = 21) and opiate use (n = 18) subjects while they were performing a cognitive task. The SNN models of EEG demonstrated different trends of dynamic clusters across the groups. This suggested to select a group of marker EEG features and resulted in an improved accuracy of EEG classification to 92%, when compared with all-feature classification. During learning of EEG data, the areas of neurons in the SNN model that form adjacent clusters (corresponding to neighboring EEG channels) were detected as fuzzy boundaries that explain overlapping activity of brain regions for each group of subjects.
An Event-based Categorization Model Using Spatio-temporal Features in a Spiking Neural Network.

Junwei Lu,Junfei Dong,Rui Yan,Huajin Tang

DOI: https://doi.org/10.1109/icaci49185.2020.9177628

2020-01-01

Abstract:This paper introduces an event-based categorization model to explore neuromorphic computing for object recognition. The proposed method makes full use of the precise timing information inherently present in the output of a bio-inspired vision sensor and utilizes event-driven processing to keep the form of address event representation (AER). In this model, a hierarchy of event-based time-surface is used to extract the spatio-temporal features of the AER data, and then these extracted features are classified by spiking neural network (SNN) with event-driven Tempotron rule. To demonstrate the effectiveness of the presented method, we conducted a series of experiments on some accessible AER datasets and compared our method to other developed event-based categorization models. Additional pooling operation in feature extraction can reduce the number of parameters and computing time. Experimental results show that our method has better classification performance and real-time computation. That is, the proposed model can extract useful features and consumes much less simulation time while still maintaining competitive accuracy.
Rapid Feedforward Computation by Temporal Encoding and Learning with Spiking Neurons

Qiang Yu,Huajin Tang,Jun Hu,Kay Chen Tan

DOI: https://doi.org/10.1109/TNNLS.2013.2245677

2017-01-01

Abstract:As we know, primates perform remarkably well in cognitive tasks such as pattern recognition. Motivated from recent findings in biological systems, a unified and consistent feedforward system network with a proper encoding scheme and supervised temporal rules is built for processing real-world stimuli. The temporal rules are used for processing the spatiotemporal patterns. To utilize these rules on images or sounds, a proper encoding method and a unified computational model with consistent and efficient learning rule are required. Through encoding, external stimuli are converted into sparse representations which also have properties of invariance. These temporal patterns are then learned through biologically derived algorithms in the learning layer, followed by the final decision presented through the readout layer. The performance of the model is also analyzed and discussed. This chapter presents a general structure of SNN for pattern recognition, showing that the SNN has the ability to learn the real-world stimuli.
New Algorithms for Encoding, Learning and Classification of Fmri Data in a Spiking Neural Network Architecture: A Case on Modeling and Understanding of Dynamic Cognitive Processes

Nikola Kasabov,Lei Zhou,Maryam Gholami Doborjeh,Zohreh Gholami Doborjeh,Jie Yang

DOI: https://doi.org/10.1109/tcds.2016.2636291

IF: 4.546

2017-01-01

IEEE Transactions on Cognitive and Developmental Systems

Abstract:This paper argues that, the third generation of neural networks-the spiking neural networks (SNNs), can be used to model dynamic, spatio-temporal, cognitive brain processes measured as functional magnetic resonance imaging (fMRI) data. This paper proposes a novel method based on the NeuCube SNN architecture for which the following new algorithms are introduced: fMRI data encoding into spike sequences; deep unsupervised learning of fMRI data in a 3-D SNN reservoir; classification of cognitive states; and connectivity visualization and analysis for the purpose of understanding cognitive dynamics. The method is illustrated on two case studies of cognitive data modeling from a benchmark fMRI data set of seeing a picture versus reading a sentence.
Pattern Recognition Computation in a Spiking Neural Network with Temporal Encoding and Learning

Qiang Yu,K. C. Tan,Huajin Tang

DOI: https://doi.org/10.1109/ijcnn.2012.6252427

2012-01-01

Abstract:Many conventional methods have been widely studied to solve the pattern recognition task, but most of them lack the biological plausibility. This paper presents a spiking neural network of integrate-and-fire neurons to perform pattern recognition. A biologically plausible supervised synaptic learning rule is used so that neurons can efficiently make a decision. The whole system contains encoding, learning and readout. It can classify complex patterns of activities stored in a vector, as well as the real-world stimuli. We test the performance of the network with digital images from the MNIST and images of alphabetic letters. It turns out to be able to classify these patterns correctly. In addition, the synaptic dynamics is shown to be compatible with many experimental observations on induction of long-term modifications, like spike-timing-dependent plasticity (STDP).
Classification and regression of spatio-temporal signals using NeuCube and its realization on SpiNNaker neuromorphic hardware

Jan Behrenbeck,Zied Tayeb,Cyrine Bhiri,Christoph Richter,Oliver Rhodes,Nikola Kasabov,Josafath I Espinosa-Ramos,Steve Furber,Gordon Cheng,Jörg Conradt

DOI: https://doi.org/10.1088/1741-2552/aafabc

Abstract:Objective: The objective of this work is to use the capability of spiking neural networks to capture the spatio-temporal information encoded in time-series signals and decode them without the use of hand-crafted features and vector-based learning and the realization of the spiking model on low-power neuromorphic hardware. Approach: The NeuCube spiking model was used to classify different grasp movements directly from raw surface electromyography signals (sEMG), the estimations of the applied finger forces as well as the classification of two motor imagery movements from raw electroencephalography (EEG). In a parallel investigation, the designed spiking decoder was implemented on SpiNNaker neuromorphic hardware, which allows low-energy real-time processing. Main results: Experimental results reveal a better classification accuracy using the NeuCube model compared to traditional machine learning methods. For sEMG classification, we reached a training accuracy of 85% and a test accuracy of 84.8%, as well as less than 19% of relative root mean square error (rRMSE) when estimating finger forces from six subjects. For the EEG classification, a mean accuracy of 75% was obtained when tested on raw EEG data from nine subjects from the existing 2b dataset from 'BCI competition IV'. Significance: This work provides a proof of concept for a successful implementation of the NeuCube spiking model on the SpiNNaker neuromorphic platform for raw sEMG and EEG decoding, which could chart a route ahead for a new generation of portable closed-loop and low-power neuroprostheses.
Navigation Learning Assessment Using EEG-Based Multi-Time Scale Spatiotemporal Compound Model

Lingling Wang,Yixin Liu,Yiqing Li,Renxiang Chen,Xiaohong Liu,Li Fu,Yao Wang

DOI: https://doi.org/10.1109/TNSRE.2023.3346766

Abstract:This study presents a novel method to assess the learning effectiveness using Electroencephalography (EEG)-based deep learning model. It is difficult to assess the learning effectiveness of professional courses in cultivating students' ability objectively by questionnaire or other assessment methods. Research in the field of brain has shown that innovation ability can be reflected from cognitive ability which can be embodied by EEG signal features. Three navigation tasks of increasing cognitive difficulty were designed and a total of 41 subjects participated in the experiment. For the classification and tracking of the subjects' EEG signals, a convolutional neural network (CNN)-based Multi-Time Scale Spatiotemporal Compound Model (MTSC) is proposed in this paper to extract and classify the features of the subjects' EEG signals. Furthermore, Spiking neural networks (SNN) -based NeuCube is used to assess the learning effectiveness and demonstrate cognitive processes, acknowledging that NeuCube is an excellent method to display the spatiotemporal differences between spikes emitted by neurons. The results of the classification experiment show that the cognitive training traces of different students in solving three navigational problems can be effectively distinguished. More importantly, new information about navigation is revealed through the analysis of feature vector visualization and model dynamics. This work provides a foundation for future research on cognitive navigation and the training of students' navigational skills.
Sparse Temporal Encoding of Visual Features for Robust Object Recognition by Spiking Neurons

Yajing Zheng,Shixin Li,Rui Yan,Huajin Tang,Kay Chen Tan

DOI: https://doi.org/10.1109/tnnls.2018.2812811

IF: 14.255

2018-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:Robust object recognition in spiking neural systems remains a challenging in neuromorphic computing area as it needs to solve both the effective encoding of sensory information and also its integration with downstream learning neurons. We target this problem by developing a spiking neural system consisting of sparse temporal encoding and temporal classifier. We propose a sparse temporal encoding algorithm which exploits both spatial and temporal information derived from an spike-timing-dependent plasticity-based HMAX feature extraction process. The temporal feature representation, thus, becomes more appropriate to be integrated with a temporal classifier based on spiking neurons rather than with nontemporal classifier. The algorithm has been validated on two benchmark data sets and the results show the temporal feature encoding and learning-based method achieves high recognition accuracy. The proposed model provides an efficient approach to perform feature representation and recognition in a consistent temporal learning framework, which is easily adapted to neuromorphic implementations.
A novel deep transfer learning framework integrating general and domain-specific features for EEG-based brain–computer interface

Zilin Liang,Zheng Zheng,Weihai Chen,Zhongcai Pei,Jianhua Wang,Jianer Chen

DOI: https://doi.org/10.1016/j.bspc.2024.106311

IF: 5.1

2024-04-19

Biomedical Signal Processing and Control

Abstract:The statistical variability of electroencephalographic (EEG) signals across individuals poses a common challenge for brain–computer interfaces (BCI). Specifically, the reuse of pre-recorded data from previous subjects presents a significant obstacle in decoding new subjects. To address these challenges, this paper introduces EEGTransferNet, an end-to-end modular transfer learning framework that enhances the similarity of statistical distributions across subjects. It utilizes convolutional neural networks as backbones for feature extraction and incorporates statistical distribution alignment and domain adaptation in different network layers to learn general features and domain-specific features respectively. The robust generalization design of EEGTransferNet enables it to handle multiple BCI paradigms using a single network making it efficient and effective. Its modular structure facilitates customization of backbone and transfer modules for diverse tasks. The optimal structure of EEGTransferNet was determined through analysis and extensive experimentation. We validated it on motor imagery (MI), error-related negativity (ERN), and rapid serial visual presentation (RSVP) datasets, which exhibits competitive performance compared to state-of-the-art methods. The experimental results confirm the effectiveness and superiority of EEGTransferNet.

engineering, biomedical
Dynamic Spatiotemporal Pattern Recognition with Recurrent Spiking Neural Network

Jiangrong Shen,Jian K. Liu,Yueming Wang

DOI: https://doi.org/10.1162/neco_a_01432

IF: 3.278

2021-08-30

Neural Computation

Abstract:Abstract Our real-time actions in everyday life reflect a range of spatiotemporal dynamic brain activity patterns, the consequence of neuronal computation with spikes in the brain. Most existing models with spiking neurons aim at solving static pattern recognition tasks such as image classification. Compared with static features, spatiotemporal patterns are more complex due to their dynamics in both space and time domains. Spatiotemporal pattern recognition based on learning algorithms with spiking neurons therefore remains challenging. We propose an end-to-end recurrent spiking neural network model trained with an algorithm based on spike latency and temporal difference backpropagation. Our model is a cascaded network with three layers of spiking neurons where the input and output layers are the encoder and decoder, respectively. In the hidden layer, the recurrently connected neurons with transmission delays carry out high-dimensional computation to incorporate the spatiotemporal dynamics of the inputs. The test results based on the data sets of spiking activities of the retinal neurons show that the proposed framework can recognize dynamic spatiotemporal patterns much better than using spike counts. Moreover, for 3D trajectories of a human action data set, the proposed framework achieves a test accuracy of 83.6% on average. Rapid recognition is achieved through the learning methodology–based on spike latency and the decoding process using the first spike of the output neurons. Taken together, these results highlight a new model to extract information from activity patterns of neural computation in the brain and provide a novel approach for spike-based neuromorphic computing.

computer science, artificial intelligence,neurosciences
Transfer learning to decode brain states reflecting the relationship between cognitive tasks

Youzhi Qu,Xinyao Jian,Wenxin Che,Penghui Du,Kai Fu,Quanying Liu

DOI: https://doi.org/10.48550/arXiv.2206.03950

2022-08-30

Abstract:Transfer learning improves the performance of the target task by leveraging the data of a specific source task: the closer the relationship between the source and the target tasks, the greater the performance improvement by transfer learning. In neuroscience, the relationship between cognitive tasks is usually represented by similarity of activated brain regions or neural representation. However, no study has linked transfer learning and neuroscience to reveal the relationship between cognitive tasks. In this study, we propose a transfer learning framework to reflect the relationship between cognitive tasks, and compare the task relations reflected by transfer learning and by the overlaps of brain regions (e.g., neurosynth). Our results of transfer learning create cognitive taskonomy to reflect the relationship between cognitive tasks which is well in line with the task relations derived from neurosynth. Transfer learning performs better in task decoding with fMRI data if the source and target cognitive tasks activate similar brain regions. Our study uncovers the relationship of multiple cognitive tasks and provides guidance for source task selection in transfer learning for neural decoding based on small-sample data.

Neurons and Cognition,Artificial Intelligence,Machine Learning
DEEP TRANSFER LEARNING FOR EEG-BASED BRAIN COMPUTER INTERFACE

Chuanqi Tan,Fuchun Sun,Tao Kong,Bin Fang,Wenchang Zhang

DOI: https://doi.org/10.1109/icassp.2018.8462115

2018-01-01

Abstract:The electroencephalography classifier is the most important component of brain-computer interface based systems. There are two major problems hindering the improvement of it. First, traditional methods do not fully exploit multimodal information. Second, large-scale annotated EEG datasets are almost impossible to acquire because biological data acquisition is challenging and quality annotation is costly. Herein, we propose a novel deep transfer learning approach to solve these two problems. First, we model cognitive events based on EEG data by characterizing the data using EEG optical flow, which is designed to preserve multimodal EEG information in a uniform representation. Second, we design a deep transfer learning framework which is suitable for transferring knowledge by joint training, which contains a adversarial network and a special loss function. The experiments demonstrate that our approach, when applied to EEG classification tasks, has many advantages, such as robustness and accuracy.
Cross-Modal Transfer Learning From EEG to Functional Near-Infrared Spectroscopy for Classification Task in Brain-Computer Interface System

Yuqing Wang,Zhiqiang Yang,Hongfei Ji,Jie Li,Lingyu Liu,Jie Zhuang

DOI: https://doi.org/10.3389/fpsyg.2022.833007

IF: 3.8

2022-04-07

Frontiers in Psychology

Abstract:The brain-computer interface (BCI) based on functional near-infrared spectroscopy (fNIRS) has received more and more attention due to its vast application potential in emotion recognition. However, the relatively insufficient investigation of the feature extraction algorithms limits its use in practice. In this article, to improve the performance of fNIRS-based BCI, we proposed a method named R-CSP-E, which introduces EEG signals when computing fNIRS signals’ features based on transfer learning and ensemble learning theory. In detail, we used the Independent Component Analysis (ICA) algorithm for the correspondence between the sources of the two signals. We then introduced the EEG signals when computing the spatial filter based on a modified Common Spatial Pattern (CSP) algorithm. Experimental results on public datasets show that the proposed method in this paper outperforms traditional methods without transfer. In general, the mean classification accuracy can be increased by up to 5%. To our knowledge, it is an innovation that we tried to apply transfer learning between EEG and fNIRS. Our study’s findings not only prove the potential of the transfer learning algorithm in cross-model brain-computer interface, but also offer a new and innovative perspective to research the hybrid brain-computer interface.

psychology, multidisciplinary
Evolving spatio-temporal data machines based on the NeuCube neuromorphic framework: Design methodology and selected applications.

Nikola Kasabov,Nathan Matthew Scott,Enmei Tu,Stefan Marks,Neelava Sengupta,Elisa Capecci,Muhaini Othman,Maryam Gholami Doborjeh,Norhanifah Murli,Reggio Hartono,Josafath Israel Espinosa-Ramos,Lei Zhou,Fahad Bashir Alvi,Grace Wang,Denise Taylor,Valery Feigin,Sergei Gulyaev,Mahmoud Mahmoud,Zeng-Guang Hou,Jie Yang

DOI: https://doi.org/10.1016/j.neunet.2015.09.011

2015-01-01

Abstract:The paper describes a new type of evolving connectionist systems (ECOS) called evolving spatio-temporal data machines based on neuromorphic, brain-like information processing principles (eSTDM). These are multi-modular computer systems designed to deal with large and fast spatio/spectro temporal data using spiking neural networks (SNN) as major processing modules. ECOS and eSTDM in particular can learn incrementally from data streams, can include 'on the fly' new input variables, new output class labels or regression outputs, can continuously adapt their structure and functionality, can be visualised and interpreted for new knowledge discovery and for a better understanding of the data and the processes that generated it. eSTDM can be used for early event prediction due to the ability of the SNN to spike early, before whole input vectors (they were trained on) are presented. A framework for building eSTDM called NeuCube along with a design methodology for building eSTDM using this is presented. The implementation of this framework in MATLAB, Java, and PyNN (Python) is presented. The latter facilitates the use of neuromorphic hardware platforms to run the eSTDM. Selected examples are given of eSTDM for pattern recognition and early event prediction on EEG data, fMRI data, multisensory seismic data, ecological data, climate data, audio-visual data. Future directions are discussed, including extension of the NeuCube framework for building neurogenetic eSTDM and also new applications of eSTDM.

Transfer Learning of Fuzzy Spatio-Temporal Rules in the NeuCube Brain-Inspired Spiking Neural Network: A Case Study on EEG Spatio-temporal Data

Transfer Learning of Fuzzy Spatio-Temporal Rules in a Brain-Inspired Spiking Neural Network Architecture: A Case Study on Spatio-Temporal Brain Data

Improving NeuCube Spiking Neural Network for EEG-based Pattern Recognition Using Transfer Learning

Mapping Temporal Variables into the NeuCube for Improved Pattern Recognition, Predictive Modeling, and Understanding of Stream Data

A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

NeuCube(ST) for spatio-temporal data predictive modelling with a case study on ecological data

NeuCube: A spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data

Deep Learning of Explainable EEG Patterns as Dynamic Spatiotemporal Clusters and Rules in a Brain-Inspired Spiking Neural Network

An Event-based Categorization Model Using Spatio-temporal Features in a Spiking Neural Network.

Rapid Feedforward Computation by Temporal Encoding and Learning with Spiking Neurons

New Algorithms for Encoding, Learning and Classification of Fmri Data in a Spiking Neural Network Architecture: A Case on Modeling and Understanding of Dynamic Cognitive Processes

Pattern Recognition Computation in a Spiking Neural Network with Temporal Encoding and Learning

Classification and regression of spatio-temporal signals using NeuCube and its realization on SpiNNaker neuromorphic hardware

Navigation Learning Assessment Using EEG-Based Multi-Time Scale Spatiotemporal Compound Model

Sparse Temporal Encoding of Visual Features for Robust Object Recognition by Spiking Neurons

A novel deep transfer learning framework integrating general and domain-specific features for EEG-based brain–computer interface

Dynamic Spatiotemporal Pattern Recognition with Recurrent Spiking Neural Network

Transfer learning to decode brain states reflecting the relationship between cognitive tasks

DEEP TRANSFER LEARNING FOR EEG-BASED BRAIN COMPUTER INTERFACE

Cross-Modal Transfer Learning From EEG to Functional Near-Infrared Spectroscopy for Classification Task in Brain-Computer Interface System

Evolving spatio-temporal data machines based on the NeuCube neuromorphic framework: Design methodology and selected applications.