Abstract:An extreme learning machine (ELM) can be regarded as a two-stage feed-forward neural network (FNN) learning system that randomly assigns the connections with and within hidden neurons in the first stage and tunes the connections with output neurons in the second stage. Therefore, ELM training is essentially a linear learning problem, which significantly reduces the computational burden. Numerous applications show that such a computation burden reduction does not degrade the generalization capability. It has, however, been open that whether this is true in theory. The aim of this paper is to study the theoretical feasibility of ELM by analyzing the pros and cons of ELM. In the previous part of this topic, we pointed out that via appropriately selected activation functions, ELM does not degrade the generalization capability in the sense of expectation. In this paper, we launch the study in a different direction and show that the randomness of ELM also leads to certain negative consequences. On one hand, we find that the randomness causes an additional uncertainty problem of ELM, both in approximation and learning. On the other hand, we theoretically justify that there also exist activation functions such that the corresponding ELM degrades the generalization capability. In particular, we prove that the generalization capability of ELM with Gaussian kernel is essentially worse than that of FNN with Gaussian kernel. To facilitate the use of ELM, we also provide a remedy to such a degradation. We find that the well-developed coefficient regularization technique can essentially improve the generalization capability. The obtained results reveal the essential characteristic of ELM in a certain sense and give theoretical guidance concerning how to use ELM.

Extreme Learning Machines with Expectation Kernels.

Extreme Learning Machine with Multiple Kernels

Multiple-kernel-learning-based Extreme Learning Machine for Classification Design

Genetic Ensemble of Extreme Learning Machine

Optimal Weighted Extreme Learning Machine for Imbalanced Learning with Differential Evolution [research Frontier]

Extreme Learning Machine Combining Hidden-Layer Feature Weighting and Batch Training for Classification

A Novel Extreme Learning Machine Based on Hybrid Kernel Function.

Mixture Correntropy-Based Kernel Extreme Learning Machines

A study on effectiveness of extreme learning machine

Optimizing Extreme Learning Machine Via Generalized Hebbian Learning and Intrinsic Plasticity Learning

A Wavelet Extreme Learning Machine

Is Extreme Learning Machine Feasible? A Theoretical Assessment (Part II)

Trends in extreme learning machines

Extreme Learning Machine with Kernel Model Based on Deep Learning

Trends in Extreme Learning Machines: A Review

Robust Matching Pursuit Extreme Learning Machines

Effective Non-Random Extreme Learning Machine

Credit Risk Evaluation with Extreme Learning Machine

Extreme learning machine: Theory and applications

A Fast Reduced Kernel Extreme Learning Machine.

Extreme Learning Machines: a Survey