Abstract:As neural interfaces become more advanced, there has been an increase in the volume and complexity of neural data recordings. These interfaces capture rich information about neural dynamics that call for efficient, real-time processing algorithms to spontaneously extract and interpret patterns of neural dynamics. Moreover, being able to do so in a fully unsupervised manner is critical as patterns in vast streams of neural data might not be easily identifiable by the human eye. Formal Deep Neural Networks (DNNs) have come a long way in performing pattern recognition tasks for various static and sequential pattern recognition applications. However, these networks usually require large labeled datasets for training and have high power consumption preventing their future embedding in active brain implants. An alternative aimed at addressing these issues are Spiking Neural Networks (SNNs) which are neuromorphic and use more biologically plausible neurons with evolving membrane potentials. In this context, we introduce here a frugal single-layer SNN designed for fully unsupervised identification and classification of multivariate temporal patterns in continuous data with a sequential approach. We show that, with only a handful number of neurons, this strategy is efficient to recognize highly overlapping multivariate temporal patterns, first on simulated data, and then on Mel Cepstral representations of speech sounds and finally on multichannel neural data. This approach relies on several biologically inspired plasticity rules, including Spike-timing-dependent plasticity (STDP), Short-term plasticity (STP) and intrinsic plasticity (IP). These results pave the way towards highly frugal SNNs for fully unsupervised and online-compatible learning of complex multivariate temporal patterns for future embedding in dedicated very-low power hardware.

Single-layer Folded RNN for Time Series Prediction and Classification under a Non-Von Neumann Architecture

Residual Recurrent Neural Networks for Learning Sequential Representations.

Dynamic Connectivity of Delay Loops in Folded-in-Time Dnn

A Clockwork RNN

Exploiting Symmetric Temporally Sparse BPTT for Efficient RNN Training

Dynamic temporal residual network for sequence modeling

Residual Memory Networks: Feed-forward approach to learn long temporal dependencies

Folded Recurrent Neural Networks for Future Video Prediction

DRRNets: Dynamic Recurrent Routing Via Low-Rank Regularization in Recurrent Neural Networks.

Nonrecurrent Neural Structure for Long-Term Dependence.

FiC-RNN: A Multi-FPGA Acceleration Framework for Deep Recurrent Neural Networks

Dual-Path RNN: Efficient Long Sequence Modeling for Time-Domain Single-Channel Speech Separation

Dropframe Scheme in Recurrent Neural Networks for Time Series Modeling

Temporally Folded Convolutional Neural Networks for Sequence Forecasting

RWKV: Reinventing RNNs for the Transformer Era

Training a multilayer dynamical spintronic network with standard machine learning tools to perform time series classification

Compact Feedforward Sequential Memory Networks for Large Vocabulary Continuous Speech Recognition

A frugal Spiking Neural Network for unsupervised classification of continuous multivariate temporal data

Efficient recurrent architectures through activity sparsity and sparse back-propagation through time

RRCNN: A novel signal decomposition approach based on recurrent residue convolutional neural network