Abstract:Spectral methods have recently emerged as a powerful tool for dimensionality reduction and manifold learning. These methods use information contained in the eigenvectors of a data affinity (i.e., item-item similarity) matrix to reveal the low dimensional structure in the high dimensional data. The most popular manifold learning algorithms include Locally Linear Embedding, ISOMAP, and Laplacian Eigenmap. However, these algorithms only provide the embedding results of training samples. There are many extensions of these approaches which try to solve the out-of-sample extension problem by seeking an embedding function in reproducing kernel Hilbert space. However, a disadvantage of all these approaches is that their computations usually involve eigen-decomposition of dense matrices which is expensive in both time and memory. In this thesis, we introduce a novel dimensionality reduction framework, called Spectral Regression (SR). SR casts the problem of learning an embedding function into a regression framework, which avoids eigen-decomposition of dense matrices. Also, with the regression as a building block, different kinds of regularizers can be naturally incorporated into our framework which makes it more flexible. SR can be performed in supervised, unsupervised and semi-supervised situation. It can make efficient use of both labeled and unlabeled points to discover the intrinsic discriminant structure in the data. We have applied our algorithms to several real world applications, e.g. face analysis, document representation and content-based image retrieval.

Sliced Inverse Regression with Adaptive Spectral Sparsity for Dimension Reduction.

Slicing-free Inverse Regression in High-dimensional Sufficient Dimension Reduction

Sparse sliced inverse regression for high dimensional data analysis

SPARSE SIR: OPTIMAL RATES AND ADAPTIVE ESTIMATION

On Consistency and Sparsity for Sliced Inverse Regression in High Dimensions

A Note on Sliced Inverse Regression with Missing Predictors.

Sliced Coordinate Analysis for Effective Dimension Reduction and Nonlinear Extensions

Sparse Sliced Inverse Regression Via Lasso

On the Structural Dimension of Sliced Inverse Regression

High Dimensional Elliptical Sliced Inverse Regression in non-Gaussian Distributions

On Sliced Inverse Regression with High-Dimensional Covariates

A Simple Approach to Online Sparse Sliced Inverse Regression

Sliced Regression for Dimension Reduction

On Spline Approximation of Sliced Inverse Regression

Dimension Reduction Via an Alternating Inverse Regression

Online Sufficient Dimension Reduction Through Sliced Inverse Regression.

Dimension Reduction in Regressions Through Weighted Variance Estimation

Spectral regression: a regression framework for efficient regularized subspace learning

Spectral Regression: A Unified Approach for Sparse Subspace Learning

On the Optimality of Sliced Inverse Regression in High Dimensions

A Slicing‐Free Perspective to Sufficient Dimension Reduction: Selective Review and Recent Developments