Abstract:An efficient optimization of network weights has been the primary goal of the artificial neural network (ANN) research community since decades. The aim of every optimization problem is to minimize the network cost which is some form of error function between the desired and the actual network outputs, during the training phase. The conventional gradient-based optimization algorithms like backpropagation are likely to get trapped in local minima and are sensitive to choices of initial weights. The evolutionary algorithms have proved their usefulness in introducing randomness into the optimization procedure, since they work on a global search strategy and induce a globally minimum solution for the network weights. In this paper, we particularly focus on ANN trained by Particle Swarm Optimization (ANN-PSO), in which the local-best and global-best particle positions represent possible solutions to the set of network weights. The global-best position of the swarm, which corresponds to the minimum cost function over time, is determined in our work by minimizing a new non-extensive cross-entropy error cost function. The non-extensive cross-entropy is derived from the non-extensive entropy with Gaussian gain that has proven to give minimum values for regular textures containing periodic information represented by uneven probability distributions. The new cross-entropy is defined, and its utility for optimizing the network weights to a globally minimum solution is analyzed in this paper. Extensive experimentation on two different versions: the baseline ANN-PSO and one of its most recent variants IOPSO-BPA, on benchmark datasets from the UCI repository, with comparisons to the state of the art, validates the efficacy of our method.

Training Method for a Feed Forward Neural Network Based on Meta-heuristics

Neural Networks Learning Using Vbest Model Particle Swarm Optimisation

A Method of Parameter Optimization for Particle Swarm Optimization Based on Stochastic Processes

The Study for Optimization of Chromatographic Condition by Means of Artificial Neural Networks

Multilayered Feed Forward Neural Network Based on Particle Swarm Optimizer Algorithm

Approach for Optimizing BackPropagation Training Based on PSO

Global-best optimization of ANN trained by PSO using the non-extensive cross-entropy with Gaussian gain

Based on Mix GA-PSO Nerve Network Algorithm

Survival Density Particle Swarm Optimization for Neural Network Training

Training RBF neural network with hybrid particle swarm optimization

Two Hybrid Learning Strategies of Feedforward Network

A New Method of Determining the Initial Weights of a Neural Network

Designing Neural Networks Using PSO-Based Memetic Algorithm

In-Depth Case Study on Artificial Neural Network Weights Optimization Using Meta-Heuristic and Heuristic Algorithmic Approach

Designing neural networks using hybrid particle swarm optimization

A New Global Optimization Algorithm for Training Feedforward Neural Networks and Its Application

Particle Swarm Optimisation In Feedforward Neural Network

Two-Input Power-Activation Neural Network Weights-Direct-Determination and Structure Optimized by Particle Swarm Optimization

Training a Feed-forward Neural Network with Artificial Bee Colony Based Backpropagation Method

Particle Swarm Optimization Based Learning Method for Process Neural Networks

An Improved Particle Swarm Optimization for Evolving Feedforward Artificial Neural Networks