Abstract:Spiking neural networks (SNNs) are believed to be a powerful neural computation framework inspired by the vivo neurons. As a class of recurrent SNNs, liquid state machines (LSMs) are biologically more plausible models imitating the architecture and functions of the human brain for information processing. However, few LSM models can outperform conventional analogue neural networks for solving real-world classification or regression problems, which can mainly be attributed to the sensitivity of the training performance to the architecture of the reservoir and the parameters in the spiking neuron models. Most recently, many algorithms have been proposed for automated machine learning that aims to automatically design the architecture and parameters of deep neural networks without much human intervention. Although automated machine learning and neural architecture search have been extremely successful in conventional neural networks, little research on search for an optimal architecture and hyperparameters of LSMs has been reported. This work proposes on a surrogate-assisted evolutionary search method for optimization of the hyperparameters and neural architecture of the reservoir of LSMs using the covariance matrix adaptation evolution strategy (CMA-ES). For reducing the search space, the architecture of the LSM is encoded by a connectivity probability together with the hyperparameters in the spiking neuron models. To enhance the computational efficiency, a Gaussian process is adopted as the surrogate to assist the CMA-ES. The proposed GP-assisted CMA-ES is compared with the canonical CMA-ES and a Bayesian optimization algorithm on two popular datasets including image and action recognition. Our results confirm that the proposed algorithm is efficient and effective in optimizing the parameters and architecture of LSMs.

The Critical Dynamics in Neural Network Improve the Computational Capability of Liquid State Machines.

Computational Capability of Liquid State Machines with Spike-Timing-dependent Plasticity.

Improving Liquid State Machine with Hybrid Plasticity

Biological Modelling of a Computational Spiking Neural Network with Neuronal Avalanches.

Increasing Liquid State Machine Performance with Edge-of-Chaos Dynamics Organized by Astrocyte-modulated Plasticity

Liquid Computing of Spiking Neural Network with Multi-Clustered and Active-Neuron-dominant Structure.

Computational Modeling of Spiking Neural Network with Learning Rules from STDP and Intrinsic Plasticity

Adaptive structure evolution and biologically plausible synaptic plasticity for recurrent spiking neural networks

Evolutionary Optimization Of Liquid State Machines For Robust Learning

Improving Liquid State Machine in Temporal Pattern Classification

Bursting Dynamics Remarkably Improve the Performance of Neural Networks on Liquid Computing

Surrogate-Assisted Cooperative Co-evolutionary Reservoir Architecture Search for Liquid State Machines

Effects of Temporal Integration on Computational Performance of Spiking Neural Network

Effects of Firing Patterns on Spike Strain Classification Performance of Liquid State Machine

Dynamic Training of Liquid State Machines

Dynamics Analysis and Analog Associative Memory of Networks with LT Neurons.

A General Description of Criticality in Neural Network Models

Effects of Synaptic Integration on the Dynamics and Computational Performance of Spiking Neural Network

Dynamic modulation of external excitation enhance synchronization in complex neuronal network

Surrogate-Assisted Evolutionary Search of Spiking Neural Architectures in Liquid State Machines