Abstract:Spiking neural networks (SNNs) are believed to be a powerful neural computation framework inspired by the vivo neurons. As a class of recurrent SNNs, liquid state machines (LSMs) are biologically more plausible models imitating the architecture and functions of the human brain for information processing. However, few LSM models can outperform conventional analogue neural networks for solving real-world classification or regression problems, which can mainly be attributed to the sensitivity of the training performance to the architecture of the reservoir and the parameters in the spiking neuron models. Most recently, many algorithms have been proposed for automated machine learning that aims to automatically design the architecture and parameters of deep neural networks without much human intervention. Although automated machine learning and neural architecture search have been extremely successful in conventional neural networks, little research on search for an optimal architecture and hyperparameters of LSMs has been reported. This work proposes on a surrogate-assisted evolutionary search method for optimization of the hyperparameters and neural architecture of the reservoir of LSMs using the covariance matrix adaptation evolution strategy (CMA-ES). For reducing the search space, the architecture of the LSM is encoded by a connectivity probability together with the hyperparameters in the spiking neuron models. To enhance the computational efficiency, a Gaussian process is adopted as the surrogate to assist the CMA-ES. The proposed GP-assisted CMA-ES is compared with the canonical CMA-ES and a Bayesian optimization algorithm on two popular datasets including image and action recognition. Our results confirm that the proposed algorithm is efficient and effective in optimizing the parameters and architecture of LSMs.

LLMatic: Neural Architecture Search via Large Language Models and Quality Diversity Optimization

LLaMA-NAS: Efficient Neural Architecture Search for Large Language Models

FL-NAS: Towards Fairness of NAS for Resource Constrained Devices via Large Language Models

Large Language Model Compression with Neural Architecture Search

Large Language Model Assisted Adversarial Robustness Neural Architecture Search

Beyond the Limit of Weight-Sharing: Pioneering Space-Evolving NAS with Large Language Models

LLM Performance Predictors are good initializers for Architecture Search

Toward Less Constrained Macro-Neural Architecture Search

Towards Less Constrained Macro-Neural Architecture Search

MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation

Search for Efficient Large Language Models

NAS-LID: Efficient Neural Architecture Search with Local Intrinsic Dimension

Efficient Multi-objective Neural Architecture Search via Lamarckian Evolution

Accelerating Neural Architecture Search for Natural Language Processing with Knowledge Distillation and Earth Mover's Distance

Multinomial Distribution Learning for Effective Neural Architecture Search

Multi-trial Neural Architecture Search with Lottery Tickets

Developing Convolutional Neural Networks using a Novel Lamarckian Co-Evolutionary Algorithm

Advances in neural architecture search

DLW-NAS: Differentiable Light-Weight Neural Architecture Search

Surrogate-Assisted Evolutionary Search of Spiking Neural Architectures in Liquid State Machines

Optimizing High-Level Synthesis Designs with Retrieval-Augmented Large Language Models