Abstract:Spectral methods have recently emerged as a powerful tool for dimensionality reduction and manifold learning. These methods use information contained in the eigenvectors of a data affinity (i.e., item-item similarity) matrix to reveal the low dimensional structure in the high dimensional data. The most popular manifold learning algorithms include Locally Linear Embedding, ISOMAP, and Laplacian Eigenmap. However, these algorithms only provide the embedding results of training samples. There are many extensions of these approaches which try to solve the out-of-sample extension problem by seeking an embedding function in reproducing kernel Hilbert space. However, a disadvantage of all these approaches is that their computations usually involve eigen-decomposition of dense matrices which is expensive in both time and memory. In this thesis, we introduce a novel dimensionality reduction framework, called Spectral Regression (SR). SR casts the problem of learning an embedding function into a regression framework, which avoids eigen-decomposition of dense matrices. Also, with the regression as a building block, different kinds of regularizers can be naturally incorporated into our framework which makes it more flexible. SR can be performed in supervised, unsupervised and semi-supervised situation. It can make efficient use of both labeled and unlabeled points to discover the intrinsic discriminant structure in the data. We have applied our algorithms to several real world applications, e.g. face analysis, document representation and content-based image retrieval.

Dimension reduction method based on spectral regression and graph regularization least square regression

Dimension Reduction of Hyperspectral Images with Sparse Linear Discriminant Analysis

A framework of joint graph embedding and sparse regression for dimensionality reduction.

Local Regularized Least-Square Dimensionality Reduction

Adaptive Sparse Graph Learning Based Dimensionality Reduction for Classification.

Sparse Unsupervised Dimensionality Reduction Algorithms

Estimating a Sparse Reduction for General Regression in High Dimensions

Spectral Regression: A Unified Approach for Sparse Subspace Learning

A Generalized Least-Squares Approach Regularized with Graph Embedding for Dimensionality Reduction

Linear Dimensionality Reduction Based on Hybrid Structure Preserving Projections

Dimension reduction for covariates in network data

Graph Regularized Sparse Coding for Image Representation

Sparse Projections over Graph.

Sparse sufficient dimension reduction using optimal scoring.

Spectral Regression Based Marginal Fisher Analysis Dimensionality Reduction Algorithm.

Compressed Spectral Regression for Efficient Nonlinear Dimensionality Reduction

Dimensionality Reduction of Hyperspectral Imagery Using Sparse Graph Learning

Dimensionality Reduction of Hyperspectral Data Based on Non-Negative Sparse Embedding Projection

Spectral regression: a regression framework for efficient regularized subspace learning

Dimensionality Reduction By Using Sparse Reconstruction Embedding

Sparsity Induced Locality Preserving Projection Approaches for Dimensionality Reduction