Abstract:An extreme learning machine (ELM) can be regarded as a two-stage feed-forward neural network (FNN) learning system that randomly assigns the connections with and within hidden neurons in the first stage and tunes the connections with output neurons in the second stage. Therefore, ELM training is essentially a linear learning problem, which significantly reduces the computational burden. Numerous applications show that such a computation burden reduction does not degrade the generalization capability. It has, however, been open that whether this is true in theory. The aim of this paper is to study the theoretical feasibility of ELM by analyzing the pros and cons of ELM. In the previous part of this topic, we pointed out that via appropriately selected activation functions, ELM does not degrade the generalization capability in the sense of expectation. In this paper, we launch the study in a different direction and show that the randomness of ELM also leads to certain negative consequences. On one hand, we find that the randomness causes an additional uncertainty problem of ELM, both in approximation and learning. On the other hand, we theoretically justify that there also exist activation functions such that the corresponding ELM degrades the generalization capability. In particular, we prove that the generalization capability of ELM with Gaussian kernel is essentially worse than that of FNN with Gaussian kernel. To facilitate the use of ELM, we also provide a remedy to such a degradation. We find that the well-developed coefficient regularization technique can essentially improve the generalization capability. The obtained results reveal the essential characteristic of ELM in a certain sense and give theoretical guidance concerning how to use ELM.

Extreme learning machine: Theory and applications

Extreme Learning Machine with Multiple Kernels

A Fast Incremental Method Based on Regularized Extreme Learning Machine

Genetic Ensemble of Extreme Learning Machine

Extreme Learning Machine: A New Learning Scheme of Feedforward Neural Networks

Multiple-kernel-learning-based Extreme Learning Machine for Classification Design

Image super-resolution representation via image patches based on extreme learning machine

Extreme learning machine: algorithm, theory and applications

A Batch Inheritance Extreme Learning Machine Algorithm Based on Regular Optimization

A study on effectiveness of extreme learning machine

Extreme Learning Machines: a Survey

A review on extreme learning machine

Extreme Learning Machines: New Trends and Applications

A Novel Extreme Learning Machine Based on Hybrid Kernel Function.

Extreme Learning Machine With Adaptive Growth Of Hidden Nodes And Incremental Updating Of Output Weights

A Fast Constructive Learning Algorithm For Single-Hidden-Layer Neural Networks

Optimizing Extreme Learning Machine Via Generalized Hebbian Learning and Intrinsic Plasticity Learning

Is Extreme Learning Machine Feasible? A Theoretical Assessment (Part II)

Distributed Extreme Learning Machine for Nonlinear Learning over Network

An Overview of Extreme Learning Machine

A Fast Training Algorithm for Extreme Learning Machine Based on Matrix Decomposition.