Abstract:The multilayer perceptron (MLP) neural network is interpreted from the geometrical viewpoint in this work, that is, an MLP partition an input feature space into multiple nonoverlapping subspaces using a set of hyperplanes, where the great majority of samples in a subspace belongs to one object class. Based on this high-level idea, we propose a three-layer feedforward MLP (FF-MLP) architecture for its implementation. In the first layer, the input feature space is split into multiple subspaces by a set of partitioning hyperplanes and rectified linear unit (ReLU) activation, which is implemented by the classical two-class linear discriminant analysis (LDA). In the second layer, each neuron activates one of the subspaces formed by the partitioning hyperplanes with specially designed weights. In the third layer, all subspaces of the same class are connected to an output node that represents the object class. The proposed design determines all MLP parameters in a feedforward one-pass fashion analytically without backpropagation. Experiments are conducted to compare the performance of the traditional backpropagation-based MLP (BP-MLP) and the new FF-MLP. It is observed that the FF-MLP outperforms the BP-MLP in terms of design time, training time, and classification performance in several benchmarking datasets. Our source code is available at https://colab.research.google.com/drive/1Gz0L8AnT4ijrUchrhEXXsnaacrFdenn?usp = sharing.

Class-modular Multi-Layer Perceptions, Task Decomposition and Virtually Balanced Training Subsets

Task Decomposition and Modular Single-Hidden-layer Perceptron Classifiers for Multi-Class Learning Problems

A Modular Single-Hidden-Layer Perceptron For Letter Recognition

Task Decomposition and Modular Perceptons with Sigmoid Activation Functions for Solving the Two-Spirals Problem

Classification Methodologies of Multilayer Perceptrons with Sigmoid Activation Functions

A Multi-Class Large Margin Classifier

Training Multilayer Perceptron with Multiple Classifier Systems

Adaptive Task Decomposition And Modular Multilayer Perceptrons For Letter Recognition

The Balanced Multi-Modal Spiking Neural Networks with Online Loss Adjustment and Time Alignment

Task Decomposition and Module Combination Based on Class Relations: a Modular Neural Network for Pattern Classification

A Mixed Parallel Perceptron Classifier and Several Application Problems

Decomposition and Parallel Learning of Imbalanced Classification Problems by Min-Max Modular Neural Network

Modular neural networks for estimating odor concentrations

Efficient Classification of Multi-Label and Imbalanced Data Using Min-Max Modular Classifiers

Learning from imbalanced data sets with a Min-Max modular support vector machine

Multitask Learning With Enhanced Modules.

Applying SP-MLP to Complex Classification Problems

A Matrix Modular Neural Network Based on Task Decomposition with Subspace Division by Adaptive Affinity Propagation Clustering

Geometrical Interpretation and Design of Multilayer Perceptrons

Methods of Training and Constructing Multilayer Perceptrons with Arbitrary Pattern Sets.

Grokking Modular Polynomials