Abstract:Artificial Neural Networks (ANNs) have been used in a wide range of applications for complex datasets with their flexible mathematical architecture. The flexibility is favored by the introduction of a higher number of connections and variables, in general. However, over-parameterization of the ANN equations and the existence of redundant input variables usually result in poor test performance. This paper proposes a superstructure-based mixed-integer nonlinear programming method for optimal structural design including neuron number selection, pruning, and input selection for multilayer perceptron (MLP) ANNs. In addition, this method uses statistical measures such as the parameter covariance matrix in order to increase the test performance while permitting reduced training performance. The suggested approach was implemented on two public hyperspectral datasets (with 10% and 50% sampling ratios), namely Indian Pines and Pavia University, for the classification problem. The test results revealed promising performances compared to the standard fully connected neural networks in terms of the estimated overall and individual class accuracies. With the application of the proposed superstructural optimization, fully connected networks were pruned by over 60% in terms of the total number of connections, resulting in an increase of 4% for the 10% sampling ratio and a 1% decrease for the 50% sampling ratio. Moreover, over 20% of the spectral bands in the Indian Pines data and 30% in the Pavia University data were found statistically insignificant, and they were thus removed from the MLP networks. As a result, the proposed method was found effective in optimizing the architectural design with high generalization capabilities, particularly for fewer numbers of samples. The analysis of the eliminated spectral bands revealed that the proposed algorithm mostly removed the bands adjacent to the pre-eliminated noisy bands and highly correlated bands carrying similar information.

Research on stereographic projection and it's application on feed forward neural network

UniGrad-FS: Unified Gradient Projection with Flatter Sharpness for Continual Learning

Investigating the Benefits of Projection Head for Representation Learning

Research on the Visual Image Processing Based on Neural Network

Some Theorems for Feed Forward Neural Networks

Learnable Subspace Orthogonal Projection for Semi-supervised Image Classification

A Projection Neural Network for Training Support Vector Machines

Improving Nonlinear Projection Heads using Pretrained Autoencoder Embeddings

Enhanced Expressive Power and Fast Training of Neural Networks by Random Projections

Hierarchical Feature Selection for Random Projection

A modified projection neural network for linear variational inequalities and quadratic optimization problems

Research on the normalization method for identification of temperature field in intelligent building based on stereographic projection

Document Classification Via TextCC Based on Stereographic Projection

Qualitative Projection Using Deep Neural Networks

Knowledge Projection for Deep Neural Networks

Billion-scale Network Embedding with Iterative Random Projection

Design of Feedforward Neural Networks in the Classification of Hyperspectral Imagery Using Superstructural Optimization

Neural Field Classifiers via Target Encoding and Classification Loss

Single-shot fringe projection profilometry based on Deep Learning and Computer Graphics

Projection based weight normalization: Efficient method for optimization on oblique manifold in DNNs

PGNN-Net: Parallel Graph Neural Networks for Hyperspectral Image Classification Using Multiple Spatial-Spectral Features