Abstract:An extreme learning machine (ELM) can be regarded as a two-stage feed-forward neural network (FNN) learning system that randomly assigns the connections with and within hidden neurons in the first stage and tunes the connections with output neurons in the second stage. Therefore, ELM training is essentially a linear learning problem, which significantly reduces the computational burden. Numerous applications show that such a computation burden reduction does not degrade the generalization capability. It has, however, been open that whether this is true in theory. The aim of this paper is to study the theoretical feasibility of ELM by analyzing the pros and cons of ELM. In the previous part of this topic, we pointed out that via appropriately selected activation functions, ELM does not degrade the generalization capability in the sense of expectation. In this paper, we launch the study in a different direction and show that the randomness of ELM also leads to certain negative consequences. On one hand, we find that the randomness causes an additional uncertainty problem of ELM, both in approximation and learning. On the other hand, we theoretically justify that there also exist activation functions such that the corresponding ELM degrades the generalization capability. In particular, we prove that the generalization capability of ELM with Gaussian kernel is essentially worse than that of FNN with Gaussian kernel. To facilitate the use of ELM, we also provide a remedy to such a degradation. We find that the well-developed coefficient regularization technique can essentially improve the generalization capability. The obtained results reveal the essential characteristic of ELM in a certain sense and give theoretical guidance concerning how to use ELM.

An Incremental Inverse-Free Augmented Complex-Valued Extreme Learning Machine

The augmented complex-valued extreme learning machine.

A Fast Incremental Method Based on Regularized Extreme Learning Machine

Multiple-kernel-learning-based Extreme Learning Machine for Classification Design

Extreme Learning Machine with Multiple Kernels

Genetic Ensemble of Extreme Learning Machine

A Batch Inheritance Extreme Learning Machine Algorithm Based on Regular Optimization

Correntropy Extreme Learning Machine Based on Spatial Pyramid Matching and Local Receptive Field

Efficient Inverse-Free Incremental and Decremental Algorithms for Multiple Hidden Nodes in Extreme Learning Machine

Online Sequential Complex-Valued ELM for Noncircular Signals: Augmented Structures and Learning Algorithms

Extreme Learning Machine Combining Hidden-Layer Feature Weighting and Batch Training for Classification

Optimizing Extreme Learning Machine Via Generalized Hebbian Learning and Intrinsic Plasticity Learning

Incremental regularized extreme learning machine and it's enhancement.

Incremental Extreme Learning Machine Based On Cascade Neural Networks

An Extreme Learning Machine Based on Artificial Immune System.

An Incremental Extreme Learning Machine for Online Sequential Learning Problems

The Mc-Elm: Learning An Elm-Like Network With Minimum Vc Dimension

Neuromorphic Extreme Learning Machines with Bimodal Memristive Synapses

An Improved Extreme Learning Machine Based on Full Rank Cholesky Factorization

Is Extreme Learning Machine Feasible? A Theoretical Assessment (Part II)

Distributed Extreme Learning Machine for Nonlinear Learning over Network