Abstract:Swarm intelligence algorithms have been widely adopted in solving many highly nonlinear, multimodal problems and have achieved tremendous successes. However, their application on deep neural networks is largely unexplored. On the other hand, deep neural networks, especially convolutional neural network (CNN), have recently achieved breakthroughs in tackling many intractable problems; nevertheless their performance depends heavily on the chosen values of their hyper-parameters, whose fine-tuning is both labor-intensive and time-consuming. In this paper, we propose a novel particle swarm optimization (PSO) variant cPSO-CNN for optimizing the hyper-parameter configuration of architecture-determined CNNs. cPSO-CNN utilizes a confidence function defined by a compound normal distribution to model experts' knowledge on CNN hyper-parameter fine-tunings so as to enhance the canonical PSO's exploration capability. cPSO-CNN also redefines the scalar acceleration coefficients of PSO as vectors to better adapt for the variant ranges of CNN hyper-parameters. Besides, a linear prediction model is adopted for fast ranking the PSO particles to reduce the cost of fitness function evaluation. The experimental results demonstrate that cPSO-CNN performs competitively when compared with several reported algorithms in terms of both CNN hyper-parameter superiority and overall computation cost.

A new stochastic PSO technique for neural network training

Enhancing Particle Swarm Optimization via probabilistic models

A Method of Parameter Optimization for Particle Swarm Optimization Based on Stochastic Processes

Survival Density Particle Swarm Optimization for Neural Network Training

PSO algorithm combined with neural network training study

A New Approach To Improve Particle Swarm Optimization

Neural Network Training Algorithm Based on Particle Swarm Optimization

A Dynamic Mutation PSO algorithm and its Application in the Neural Networks

Cpso-Cnn: an Efficient PSO-based Algorithm for Fine-Tuning Hyper-Parameters of Convolutional Neural Networks.

Particle Swarm Optimization with Moving Particles on Scale-Free Networks

Cooperative Stochastic Particle Swarm Optimizer for Neural Network Prediction Modeling

A Naive Particle Swarm Optimization

Fitness and Diversity Guided Particle Swarm Optimization for Global Optimization and Training Artificial Neural Networks

Training Recurrent Neutral Networks Based On A Novel Adaptive Particle Swarm Optimization Algorithm

PSO-PS: Parameter Synchronization with Particle Swarm Optimization for Distributed Training of Deep Neural Networks

Development of Particle Swarm Optimization Algorithm

A Cooperative Approach to Particle Swarm Optimization

Training RBF neural network with hybrid particle swarm optimization

A Dynamic Perception Coefficients Self-tuning Particle Swarm Optimization Algorithm Based on Nonlinear Systems

Stochastic Configuration Networks with Particle Swarm Optimisation Search

A stochastic scattering particle swarm optimizer