Abstract:Extreme learning machine (ELM) has been intensively studied during the last decade due to its high efficiency, effectiveness and easy-to-implementation. Recently, many variants, such as parallel ELM (P-ELM) incremental ELM and online sequential ELM(OS-ELM), have been proposed to improve its timing performance and enable its ability of incremental learning. In this paper, we propose two parallel variants, termed as data parallel regularized ELM (DPR-ELM) and model parallel regularized ELM (MPR-ELM), to further improve the computational efficiency of ELM in handling large scale learning tasks. Collectively, these two variants are called as parallel regularized ELM (PR-ELM). Specifically, our proposed algorithms are implemented on cluster with Message Passing Interface (MPI) environment. In summary, the advantages of the proposed PR-ELM algorithms over existing variants are highlighted as follows: (1) They have better parallelism since they train each data block or each sub-model independently. (2) They dramatically reduce the requirement of huge runtime memory since the whole datasets or the whole model are split into small chunks or sub-models. (3) Both DPR-ELM and MPR-ELM have better scalability since they are able to be configured on clusters with many more computing nodes. Extensive experiments have been conducted to validate the effectiveness of the proposed algorithms. As shown, DPR-ELM and MPR-ELM achieve 5.15× and 3.5× speedup on cluster with six nodes, respectively. Moreover, the speedup of DPR-ELM increases to 5.85× with the increase of the size of dataset, and this quantity is increased to 4× for MPR-ELM with the increase of the number of hidden nodes.

A Pruning Ensemble Model of Extreme Learning Machine with <Emphasis Type="Italic">L</Emphasis><Subscript>1/2</Subscript> Regularizer

A Pruning Ensemble Model of Extreme Learning Machine with $$L_{1/2}$$L1/2 Regularizer

Genetic Ensemble of Extreme Learning Machine

A Fast Incremental Method Based on Regularized Extreme Learning Machine

A pruning algorithm with L 1/2 regularizer for extreme learning machine

Hierarchical Pruning Discriminative Extreme Learning Machine

PR-ELM: Parallel regularized extreme learning machine based on cluster.

Pr-elm

Optimizing Extreme Learning Machine Via Generalized Hebbian Learning and Intrinsic Plasticity Learning

L1-Norm Robust Regularized Extreme Learning Machine with Asymmetric C-Loss for Regression.

Joint Sparse Regularization Based Sparse Semi-Supervised Extreme Learning Machine (S3ELM) for Classification

Incremental regularized extreme learning machine and it's enhancement.

Dynamic Adjustment of Hidden Node Parameters for Extreme Learning Machine

Parsimonious Extreme Learning Machine Using Recursive Orthogonal Least Squares.

Ensemble Delta Test-Extreme Learning Machine (Dt-Elm) for Regression

Training an Extreme Learning Machine by Localized Generalization Error Model

An optimization method of extreme learning machine for regression.

Evolutionary Selection Extreme Learning Machine Optimization for Regression.

LASSO EXTREME LEARNING MACHINE

A robust double-parallel extreme learning machine based on an improved M-estimation algorithm

Double parallel feedforward neural network based on extreme learning machine with L 1/2 regularizer