Abstract:Swarm intelligence algorithms have been widely adopted in solving many highly nonlinear, multimodal problems and have achieved tremendous successes. However, their application on deep neural networks is largely unexplored. On the other hand, deep neural networks, especially convolutional neural network (CNN), have recently achieved breakthroughs in tackling many intractable problems; nevertheless their performance depends heavily on the chosen values of their hyper-parameters, whose fine-tuning is both labor-intensive and time-consuming. In this paper, we propose a novel particle swarm optimization (PSO) variant cPSO-CNN for optimizing the hyper-parameter configuration of architecture-determined CNNs. cPSO-CNN utilizes a confidence function defined by a compound normal distribution to model experts' knowledge on CNN hyper-parameter fine-tunings so as to enhance the canonical PSO's exploration capability. cPSO-CNN also redefines the scalar acceleration coefficients of PSO as vectors to better adapt for the variant ranges of CNN hyper-parameters. Besides, a linear prediction model is adopted for fast ranking the PSO particles to reduce the cost of fitness function evaluation. The experimental results demonstrate that cPSO-CNN performs competitively when compared with several reported algorithms in terms of both CNN hyper-parameter superiority and overall computation cost.

Particle Swarm Optimization with Pre-Crossover and Its Application in Soft-Sensor of C3 Concentration of FCCU

Modeling soft sensor based on support vector machine and particle swarm optimization algorithms

An Improved Particle Swarm Algorithm and Its Application in Soft Sensor Modeling

Soft Sensing Model of C3 Concentration of FCCU Based on PSO-BP Neural Network

Improved Particle Swarm Optimization Algorithms by Alopex and Its Application in Soft Sensor Modeling

Improved Particle Swarm Optimization and Application in Soft Sensor Modeling

A new particle swarm optimization and the application in the soft sensor modeling

An Improved Particle Swarm Optimizer with Shuffled Sub-Swarms and Its Application in Soft-Sensor of Gasoline Endpoint

Particle swarm optimizer with crossover operation.

Improved particle swarm optimization algorithm and its application in soft sensor modeling

Soft Sensor for Ammonia Concentration at the Ammonia Converter Outlet Based on an Improved Particle Swarm Optimization and BP Neural Network

Optimization of a Fluid Catalytic Cracking Kinetic Model by Improved Particle Swarm Optimization

Research on hybrid strategy Particle Swarm Optimization algorithm and its applications

Cpso-Cnn: an Efficient PSO-based Algorithm for Fine-Tuning Hyper-Parameters of Convolutional Neural Networks.

Fuzzy Neural Network Based on Improved Cloud Particle Swarm Optimization and Its Application in the Soft Sensing of Methanol Synthesis Tower Conversion

A hybrid particle swarm optimization with crisscross learning strategy

Improved Cooperative Particle Swarm Optimizer for Design of Fuzzy Neural Network Control System

On Convergence And Parameter Selection Of An Improved Particle Swarm Optimization

An Adaptive Convergence Speed Controller Framework For Particle Swarm Optimization Variants In Single Objective Optimization Problems

PSO algorithm combined with neural network training study

Knowledge-based cooperative particle swarm optimization