Abstract:Learning sparse representations for deep networks has drawn considerable research interest in recent years. In this paper, we present a novel framework to learn sparse representations via a generalized encoder-decoder architecture. The basic idea is to adopt a fast approximation to the iterative sparse coding solution and form an efficient nonlinear encoder to map an input to a sparse representation. A set of basis functions is then learned through the minimization of an energy function consisting of a sparseness prior and linear decoder constraints. Applying a greedy layer-wise learning scheme, this framework can be extended to more layers to learn deep networks. The proposed learning algorithm is also highly efficient as no iterative operations are required, and both batch and on-line learning are supported. Given the sparse representation and basis functions, an optimized decoding procedure is carried out to reconstruct and denoise the input signals. We evaluate our model on natural image patches to develop a dictionary of V1-like Gabor filters, and further show that basis functions in a higher layer (e.g., V2) combine the filters in a lower layer to generate more complex patterns to benefit the high-level tasks. We then use the sparse representations to recognize objects in two benchmark data sets (i.e., CIFAR-10 and NORB) via a linear SVM classifier, and demonstrate better or comparable recognition performances with respect to state-of-art algorithms. The image reconstruction of MNIST images and the restoration of corrupted versions are presented at the end.

Pseudoinverse Learning Algorithom for Fast Sparse Autoencoder Training.

Broad and Pseudoinverse Learning for Autoencoder

Autoencoder, Low Rank Approximation and Pseudoinverse Learning Algorithm.

DensePILAE: a feature reuse pseudoinverse learning algorithm for deep stacked autoencoder

Fast Inverse-Free Generalized Sparse Bayesian Iearning Algorithm

PILAE: A Non-gradient Descent Learning Scheme for Deep Feedforward Neural Networks

Fast Image Recognition with Gabor Filter and Pseudoinverse Learning AutoEncoders

Pseudoinverse Learners - New Trend and Applications to Big Data.

A Vest of the Pseudoinverse Learning Algorithm.

Review of Pseudoinverse Learning Algorithm for Multilayer Neural Networks and Applications.

A NEW TRAINING ALGORITHM OF AUTOENCODER COMBINING LATERAL INHIBITION MECHANISM

Learning Sparse Representation Via a Nonlinear Shrinkage Encoder and a Linear Sparse Decoder.

Boosting and Residual Learning Scheme with Pseudoinverse Learners

A Pseudoinverse Incremental Algorithm For Fast Training Deep Neural Networks With Application To Spectra Pattern Recognition

A pseudoinverse learning algorithm for feedforward neural networks with stacked generalization applications to software reliability growth data

An Efficient Learning Algorithm for Direct Training Deep Spiking Neural Networks

Discriminative Recurrent Sparse Auto-Encoders

Learning Deep ℓ_0 Encoders

Learning Deep &#8467;<sub>0</sub> Encoders

HSAE: A Hessian Regularized Sparse Auto-Encoders.

Kernel Principal Component Analysis Pseudoinverse Learning Algorithm