Abstract:An extreme learning machine (ELM) can be regarded as a two-stage feed-forward neural network (FNN) learning system that randomly assigns the connections with and within hidden neurons in the first stage and tunes the connections with output neurons in the second stage. Therefore, ELM training is essentially a linear learning problem, which significantly reduces the computational burden. Numerous applications show that such a computation burden reduction does not degrade the generalization capability. It has, however, been open that whether this is true in theory. The aim of this paper is to study the theoretical feasibility of ELM by analyzing the pros and cons of ELM. In the previous part of this topic, we pointed out that via appropriately selected activation functions, ELM does not degrade the generalization capability in the sense of expectation. In this paper, we launch the study in a different direction and show that the randomness of ELM also leads to certain negative consequences. On one hand, we find that the randomness causes an additional uncertainty problem of ELM, both in approximation and learning. On the other hand, we theoretically justify that there also exist activation functions such that the corresponding ELM degrades the generalization capability. In particular, we prove that the generalization capability of ELM with Gaussian kernel is essentially worse than that of FNN with Gaussian kernel. To facilitate the use of ELM, we also provide a remedy to such a degradation. We find that the well-developed coefficient regularization technique can essentially improve the generalization capability. The obtained results reveal the essential characteristic of ELM in a certain sense and give theoretical guidance concerning how to use ELM.

The Extreme Learning Machine Learning Algorithm with Tunable Activation Function.

Genetic Ensemble of Extreme Learning Machine

Extreme Learning Machine with Multiple Kernels

Extreme Learning Machine Based Transfer Learning for Data Classification.

Multiple-kernel-learning-based Extreme Learning Machine for Classification Design

Extreme Learning Machine Via Free Sparse Transfer Representation Optimization

Fast Sparse Approximation of Extreme Learning Machine

A Batch Inheritance Extreme Learning Machine Algorithm Based on Regular Optimization

Optimal Weighted Extreme Learning Machine for Imbalanced Learning with Differential Evolution [research Frontier]

Optimizing Extreme Learning Machine Via Generalized Hebbian Learning and Intrinsic Plasticity Learning

An Extreme Learning Machine Based on Artificial Immune System.

Extreme Learning Machine Combining Hidden-Layer Feature Weighting and Batch Training for Classification

Tuning extreme learning machine by an improved electromagnetism-like mechanism algorithm for classification problem.

A study on effectiveness of extreme learning machine

Dynamic Adjustment of Hidden Node Parameters for Extreme Learning Machine

Is Extreme Learning Machine Feasible? A Theoretical Assessment (Part II)

Evolutionary Selection Extreme Learning Machine Optimization for Regression.

Extreme learning machine: Theory and applications

A Robust Extreme Learning Machine Based on Adaptive Loss Function for Regression Modeling

Effective Non-Random Extreme Learning Machine

Robust Extreme Learning Machine Using New Activation and Loss Functions Based on M-Estimation for Regression and Classification