Abstract:Feedforward neural network (FNN) is one of the most widely used and fastest-developed artificial neural networks. Much evolutionary computation (EC) methods have been used to optimize the weights of FNN. However, as the dimension of datasets increases, the number of weights also increases dramatically. On high-dimensional datasets, if EC methods are used directly to optimize the weights of FNN, it is impossible to obtain the optimal weights of the FNN by EC methods in an acceptable time. Feature selection is a method that can effectively reduce the computational complexity of FNN by reducing irrelevant and redundant features. It may be practical to optimize the FNN by EC methods if we first employ the feature selection for the large-scale datasets. In this paper, we present a self-adaptive parameter and strategy-based particle swarm optimization (SPS-PSO) algorithm to optimize FNN with feature selection. First, we propose an optimization model for FNN by transforming the designing of FNN into a weights optimization problem. Simultaneously, we present a feature selection optimization model. Second, we present an SPS-PSO algorithm. In this algorithm, we use the parameter and strategy self-adaptive mechanism. In addition, five candidate solution generating strategies (CSGS) are used. The experiments are divided into two groups. In the first group, SPS-PSO and three other EC methods are used to directly optimize the weights of FNN on eight datasets without any modification. In the second group, we first employ SPS-PSO-based feature selection on the original datasets and obtain eight relatively smaller datasets with the k-nearest neighbor (KNN) which is used as the evaluation function for saving time. Then, we use the new datasets as the inputs for FNN. We optimize the weights of FNN again by SPS-PSO and three other EC methods to investigate whether we can get similar or even better classification accuracy by comparing the results with that of the first group. The experimental results show that SPS-PSO has the advantage in optimizing the weights of FNN compared with the other EC methods. Meanwhile, the SPS-PSO-based feature selection can reduce the solution size and computational complexity while ensuring the classification accuracy when it is used to preprocess the datasets for FNN. In this method, a solution with an originally higher than 700 000 dimensions can be even reduced to hundreds of dimensions.

Weight Initialization of Feedforward Neural Networks by Means of Partial Least Squares

Soft-sensor modeling via neural network PLS approach

Dynamic node creation and fast learning algorithm for a hybrid feedforward neural network

Optimal Weighting Distance-Based Similarity for Locally Weighted PLS Modeling

A Deep Belief Network with PLSR for Nonlinear System Modeling

Deep PLS: A Lightweight Deep Learning Model for Interpretable and Efficient Data Analytics

A weight initialization based on the linear product structure for neural networks

A weight initialization method for fuzzy neural network based on rule partition

Improvement and Application on Back Propagation Network Based on Partial Least-squares Algorithm

A Closed-form Solution for Weight Optimization in Fully-connected Feed-forward Neural Networks

Deep Learning Partial Least Squares

Optimal Algorithm for a Multilayer Feedforward Net and Its Application

Large-Scale Feedforward Neural Network Optimization by a Self-Adaptive Strategy and Parameter Based Particle Swarm Optimization.

A novel feature fusion method based on partial least squares regression

A Novel Global Hybrid Algorithm for Feedforward Neural Networks

A mathematical framework for improved weight initialization of neural networks using Lagrange multipliers

Hybrid Fuzzy Polynomial Neural Networks with the Aid of Weighted Fuzzy Clustering Method and Fuzzy Polynomial Neurons

A New Method of Determining the Initial Weights of a Neural Network

Study on the Method of Nonlinear Partial Least Squares and Its Applications

A Novel Hybrid Evolutionary Algorithm Based on Pso and Afsa for Feedforward Neural Network Training

A Fast Feedforward Small-World Neural Network for Nonlinear System Modeling