Abstract:Artificial Neural Networks (ANNs) have been used in a wide range of applications for complex datasets with their flexible mathematical architecture. The flexibility is favored by the introduction of a higher number of connections and variables, in general. However, over-parameterization of the ANN equations and the existence of redundant input variables usually result in poor test performance. This paper proposes a superstructure-based mixed-integer nonlinear programming method for optimal structural design including neuron number selection, pruning, and input selection for multilayer perceptron (MLP) ANNs. In addition, this method uses statistical measures such as the parameter covariance matrix in order to increase the test performance while permitting reduced training performance. The suggested approach was implemented on two public hyperspectral datasets (with 10% and 50% sampling ratios), namely Indian Pines and Pavia University, for the classification problem. The test results revealed promising performances compared to the standard fully connected neural networks in terms of the estimated overall and individual class accuracies. With the application of the proposed superstructural optimization, fully connected networks were pruned by over 60% in terms of the total number of connections, resulting in an increase of 4% for the 10% sampling ratio and a 1% decrease for the 50% sampling ratio. Moreover, over 20% of the spectral bands in the Indian Pines data and 30% in the Pavia University data were found statistically insignificant, and they were thus removed from the MLP networks. As a result, the proposed method was found effective in optimizing the architectural design with high generalization capabilities, particularly for fewer numbers of samples. The analysis of the eliminated spectral bands revealed that the proposed algorithm mostly removed the bands adjacent to the pre-eliminated noisy bands and highly correlated bands carrying similar information.

Error Feature Analysis of Multilayered Feedforward Neural Networks

Robustness Analysis of Feedforward Neural Networks Composed of Threshold Neurons

Learning the Error Features of Approximate Multipliers for Neural Network Applications

A novel compound neural network for fault diagnosis

Neural Networks: Further Insights into Error Function, Generalized Weights and Others

Faulty behavior of feedforward neural networks

The superior fault tolerance of artificial neural network training with a fault/noise injection-based genetic algorithm

Novel Neural Networks for Eliminating Errors Existing in Fog

Fault-Tolerant Analysis of Feedback Neural Networks with Threshold Neurons.

Automatic Implementation of Neural Networks through Reaction Networks--Part II: Error Analysis

Design of Feedforward Neural Networks in the Classification of Hyperspectral Imagery Using Superstructural Optimization

Convergence and objective functions of noise-injected multilayer perceptrons with hidden multipliers

An Analysis of Weight Decay As a Methodology of Reducing Three-Layer Feedforward Artificial Neural Networks for Classification Problems

A Neural Network Approach to Fault Diagnosis in Analog Circuits.

Strategy for Identifying Analog Circuit Faults Using Improved Neural Network Algorithms

SPFQ: A Stochastic Algorithm and Its Error Analysis for Neural Network Quantization

The Faulty Behavior of Feedforward Neural Networks with Hard-Limiting Activation Function

Error analysis for hybrid finite element/neural network discretizations

Full error analysis for the training of deep neural networks

A Fault Tolerant Design and Reliability Evaluation Method for Artificial Neural Network Controller

Structural Reliability Analysis Using Adaptive Artificial Neural Networks