Abstract:Spiking neural networks (SNNs) are believed to be a powerful neural computation framework inspired by the vivo neurons. As a class of recurrent SNNs, liquid state machines (LSMs) are biologically more plausible models imitating the architecture and functions of the human brain for information processing. However, few LSM models can outperform conventional analogue neural networks for solving real-world classification or regression problems, which can mainly be attributed to the sensitivity of the training performance to the architecture of the reservoir and the parameters in the spiking neuron models. Most recently, many algorithms have been proposed for automated machine learning that aims to automatically design the architecture and parameters of deep neural networks without much human intervention. Although automated machine learning and neural architecture search have been extremely successful in conventional neural networks, little research on search for an optimal architecture and hyperparameters of LSMs has been reported. This work proposes on a surrogate-assisted evolutionary search method for optimization of the hyperparameters and neural architecture of the reservoir of LSMs using the covariance matrix adaptation evolution strategy (CMA-ES). For reducing the search space, the architecture of the LSM is encoded by a connectivity probability together with the hyperparameters in the spiking neuron models. To enhance the computational efficiency, a Gaussian process is adopted as the surrogate to assist the CMA-ES. The proposed GP-assisted CMA-ES is compared with the canonical CMA-ES and a Bayesian optimization algorithm on two popular datasets including image and action recognition. Our results confirm that the proposed algorithm is efficient and effective in optimizing the parameters and architecture of LSMs.

Adaptive structure evolution and biologically plausible synaptic plasticity for recurrent spiking neural networks

Adaptive structure evolution and biologically plausible synaptic plasticity for recurrent spiking neural networks

Computational Modeling of Spiking Neural Network with Learning Rules from STDP and Intrinsic Plasticity

Surrogate-Assisted Cooperative Co-evolutionary Reservoir Architecture Search for Liquid State Machines

Increasing Liquid State Machine Performance with Edge-of-Chaos Dynamics Organized by Astrocyte-modulated Plasticity

Improving Liquid State Machine with Hybrid Plasticity

Surrogate-Assisted Evolutionary Search of Spiking Neural Architectures in Liquid State Machines

Evolutionary Optimization Of Liquid State Machines For Robust Learning

Emergence of brain-inspired small-world spiking neural network through neuroevolution

The Critical Dynamics in Neural Network Improve the Computational Capability of Liquid State Machines.

A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

Computational Capability of Liquid State Machines with Spike-Timing-dependent Plasticity.

Biologically Inspired Structure Learning with Reverse Knowledge Distillation for Spiking Neural Networks

Brain-inspired Evolutionary Architectures for Spiking Neural Networks

Toward Efficient Processing and Learning with Spikes: New Approaches for Multispike Learning

Spike-driven Multi-Scale Learning with Hybrid Mechanisms of Spiking Dendrites.

Learning the Plasticity: Plasticity-Driven Learning Framework in Spiking Neural Networks

Learning by Stimulation Avoidance: A Principle to Control Spiking Neural Networks Dynamics

Beyond spiking networks: The computational advantages of dendritic amplification and input segregation

Enhancing learning in spiking neural networks through neuronal heterogeneity and neuromodulatory signaling

Two-compartment neuronal spiking model expressing brain-state specific apical-amplification, -isolation and -drive regimes