Abstract:An efficient optimization of network weights has been the primary goal of the artificial neural network (ANN) research community since decades. The aim of every optimization problem is to minimize the network cost which is some form of error function between the desired and the actual network outputs, during the training phase. The conventional gradient-based optimization algorithms like backpropagation are likely to get trapped in local minima and are sensitive to choices of initial weights. The evolutionary algorithms have proved their usefulness in introducing randomness into the optimization procedure, since they work on a global search strategy and induce a globally minimum solution for the network weights. In this paper, we particularly focus on ANN trained by Particle Swarm Optimization (ANN-PSO), in which the local-best and global-best particle positions represent possible solutions to the set of network weights. The global-best position of the swarm, which corresponds to the minimum cost function over time, is determined in our work by minimizing a new non-extensive cross-entropy error cost function. The non-extensive cross-entropy is derived from the non-extensive entropy with Gaussian gain that has proven to give minimum values for regular textures containing periodic information represented by uneven probability distributions. The new cross-entropy is defined, and its utility for optimizing the network weights to a globally minimum solution is analyzed in this paper. Extensive experimentation on two different versions: the baseline ANN-PSO and one of its most recent variants IOPSO-BPA, on benchmark datasets from the UCI repository, with comparisons to the state of the art, validates the efficacy of our method.

Neural Networks Learning Using Vbest Model Particle Swarm Optimisation

Asynchronous particle swarm optimizer with relearning strategy

Enhancing Particle Swarm Optimization via probabilistic models

An Improved Particle Swarm Optimization Algorithm with Neighborhoods Topologies

Evolving Neural Networks Using the Hybrid of Ant Colony Optimization and Bp Algorithms

Application of Artificial Neural Network and Particle Swarm Optimization in Car-Following Model

Research on Neural Networks Learning Algorithm Based on Pso and Coa

Designing neural networks using hybrid particle swarm optimization

Particle swarm optimization performance improvement using deep learning techniques

A Modified Particle Swarm Optimization Algorithm for Global Optimizations of Inverse Problems

Particle Swarm Optimization Based Learning Method for Process Neural Networks

A Particle Swarm Optimization Method Applied to Global Optimization of Inverse Problem

A K-means-based Multi-subpopulation Particle Swarm Optimization for Neural Network Ensemble

A Controlling-Chaos Method on the Basis of Particle Swarm Optimization Algorithm

Learning Bayesian Network Structures with Discrete Particle Swarm Optimization Algorithm.

Learning Bayesian Networks from Data by Particle Swarm Optimization

Global-best optimization of ANN trained by PSO using the non-extensive cross-entropy with Gaussian gain

A Particle-based Sparse Gaussian Process Optimizer

Designing Neural Networks Using PSO-Based Memetic Algorithm

The Backpropagation Artificial Neural Network Based on Elite Particle Swam Optimization Algorithm for Stochastic Linear Bilevel Programming Problem

Hyper-parameter selection in deep neural networks using parallel particle swarm optimization