Abstract:The random-hidden-node extreme learning machine (ELM) is a much more generalized cluster of single-hidden-layer feed-forward neural networks (SLFNs) which has three parts: random projection, non-linear transformation, and ridge regression (RR) model. Networks with deep architectures have demonstrated state-of-the-art performance in a variety of settings, especially with computer vision tasks. Deep learning algorithms such as stacked autoencoder (SAE) and deep belief network (DBN) are built on learning several levels of representation of the input. Beyond simply learning features by stacking autoencoders (AE), there is a need for increasing its robustness to noise and reinforcing the sparsity of weights to make it easier to discover interesting and prominent features. The sparse AE and denoising AE was hence developed for this purpose. This paper proposes an approach: SSDAE-RR (stacked sparse denoising autoencoder – ridge regression) that effectively integrates the advantages in SAE, sparse AE, denoising AE, and the RR implementation in ELM algorithm. We conducted experimental study on real-world classification (binary and multiclass) and regression problems with different scales among several relevant approaches: SSDAE-RR, ELM, DBN, neural network (NN), and SAE. The performance analysis shows that the SSDAE-RR tends to achieve a better generalization ability on relatively large datasets (large sample size and high dimension) that were not pre-processed for feature abstraction. For 16 out of 18 tested datasets, the performance of SSDAE-RR is more stable than other tested approaches. We also note that the sparsity regularization and denoising mechanism seem to be mandatory for constructing interpretable feature representations. The fact that a SSDAE-RR approach often has a comparable training time to ELM makes it useful in some real applications.

Diversified feature representation via deep auto-encoder ensemble through multiple activation functions

Exploiting SDAE Model for Recommendations

Denoising Auto-Encoders Toward Robust Unsupervised Feature Representation.

Diverse Activation Functions in Deep Learning

Reconstruction of Hidden Representation for Robust Feature Extraction

BROAD AUTOENCODER FEATURES LEARNING FOR PATTERN CLASSIFICATION PROBLEMS

A new Stack Autoencoder: Neighbouring Sample Envelope Embedded Stack Autoencoder Ensemble Model

Physiological Feature Based Emotion Recognition Via an Ensemble Deep Autoencoder with Parsimonious Structure

Representation learning via an integrated autoencoder for unsupervised domain adaptation

An Adaptive Image Denoising Method Based on Deep Rectified Denoising Auto-Encoder

Auto-Ensemble: An Adaptive Learning Rate Scheduling based Deep Learning Model Ensembling

Contractive De-noising Auto-encoder

Building Feature Space of Extreme Learning Machine with Sparse Denoising Stacked-Autoencoder.

Multi-local Collaborative AutoEncoder

Efficient Diversity-Driven Ensemble for Deep Neural Networks

Hybrid Embedded Deep Stacked Sparse Autoencoder with w_LPPD SVM Ensemble

AENEA: A novel autoencoder-based network embedding algorithm

Embedding with Autoencoder Regularization.

Stacked Convolutional Sparse Auto-Encoders for Representation Learning

Denoising Deep Extreme Learning Machine for Sparse Representation

Stacked Denoising Extreme Learning Machine Autoencoder Based on Graph Embedding for Feature Representation