Abstract:Linear representation learning is widely studied due to its conceptual simplicity and empirical utility in tasks such as compression, classification, and feature extraction. Given a set of points $[\mathbf{x}_1, \mathbf{x}_2, \ldots, \mathbf{x}_n] = \mathbf{X} \in \mathbb{R}^{d \times n}$ and a vector $\mathbf{y} \in \mathbb{R}^d$, the goal is to find coefficients $\mathbf{w} \in \mathbb{R}^n$ so that $\mathbf{X} \mathbf{w} \approx \mathbf{y}$, subject to some desired structure on $\mathbf{w}$. In this work we seek $\mathbf{w}$ that forms a local reconstruction of $\mathbf{y}$ by solving a regularized least squares regression problem. We obtain local solutions through a locality function that promotes the use of columns of $\mathbf{X}$ that are close to $\mathbf{y}$ when used as a regularization term. We prove that, for all levels of regularization and under a mild condition that the columns of $\mathbf{X}$ have a unique Delaunay triangulation, the optimal coefficients' number of non-zero entries is upper bounded by $d+1$, thereby providing local sparse solutions when $d \ll n$. Under the same condition we also show that for any $\mathbf{y}$ contained in the convex hull of $\mathbf{X}$ there exists a regime of regularization parameter such that the optimal coefficients are supported on the vertices of the Delaunay simplex containing $\mathbf{y}$. This provides an interpretation of the sparsity as having structure obtained implicitly from the Delaunay triangulation of $\mathbf{X}$. We demonstrate that our locality regularized problem can be solved in comparable time to other methods that identify the containing Delaunay simplex.

Locality Regularized Reconstruction: Structured Sparsity and Delaunay Triangulations

Locality Pursuit Embedding

Sparsity Induced Locality Preserving Projection Approaches for Dimensionality Reduction

Regularized Locality Preserving Projections with Two-Dimensional Discretized Laplacian Smoothing

Compressive Sensing Via Nonlocal Low-Rank Regularization

Locally Regularized Sparse Graph by Fast Proximal Gradient Descent

Local Feature Discriminant Projection

The Geometry and Well-Posedness of Sparse Regularized Linear Regression

Local and Global Regularized Sparse Coding for Data Representation

Sparse Norm-Regularized Reconstructive Coefficients Learning

Multi-Objective Sparse Reconstruction With Transfer Learning and Localized Regularization

A Structural Low Rank Regularization Method for Single Image Super-Resolution

Efficient Nonconvex Regularized Tensor Completion with Structure-aware Proximal Iterations

Non-line-of-sight reconstruction via structure sparsity regularization

Learning Sparse Visual Representations with Leaky Capped Norm Regularizers

Reconstruction Regularized Low-Rank Subspace Learning for Cross-Modal Retrieval

Low-Rank Preserving Projection Via Graph Regularized Reconstruction

Laplacian Regularized Low-Rank Representation and Its Applications

Structured Regularization for Constrained Optimization on the SPD Manifold

Discriminative Subspace Learning With Adaptive Graph Regularization

High-dimensional Inference Via Lipschitz Sparsity-Yielding Regularizers.