Abstract:Recently there is a line of research work proposing to employ Spectral Clustering (SC) to segment (group)\footnote{Throughout the paper, we use segmentation, clustering, and grouping, and their verb forms, interchangeably.} high-dimensional structural data such as those (approximately) lying on subspaces\footnote{We follow~\cite{liu2010robust} and use the term ``subspace'' to denote both linear subspaces and affine subspaces. There is a trivial conversion between linear subspaces and affine subspaces as mentioned therein.} or low-dimensional manifolds. By learning the affinity matrix in the form of sparse reconstruction, techniques proposed in this vein often considerably boost the performance in subspace settings where traditional SC can fail. Despite the success, there are fundamental problems that have been left unsolved: the spectrum property of the learned affinity matrix cannot be gauged in advance, and there is often one ugly symmetrization step that post-processes the affinity for SC input. Hence we advocate to enforce the symmetric positive semi definite constraint explicitly during learning (Low-Rank Representation with Positive Semi Definite constraint, or LRR-PSD), and show that factually it can be solved in an exquisite scheme efficiently instead of general-purpose SDP solvers that usually scale up poorly. We provide rigorous mathematical derivations to show that, in its canonical form, LRR-PSD is equivalent to the recently proposed Low-Rank Representation (LRR) scheme~\cite{liu2010robust}, and hence offer theoretic and practical insights to both LRR-PSD and LRR, inviting future research. As per the computational cost, our proposal is at most comparable to that of LRR, if not less. We validate our theoretic analysis and optimization scheme by experiments on both synthetic and real data sets.

Robust Learning from Noisy Side-information by Semidefinite Programming

Efficient Low-Rank Semidefinite Programming With Robust Loss Functions

A Dual Perspective of Sparse and Robust Online Learning Algorithm

On semidefinite programming relaxations for a class of robust SOS-convex polynomial optimization problems

Safety Verification and Robustness Analysis of Neural Networks via Quadratic Constraints and Semidefinite Programming

Learning-Augmented Algorithms for Online Linear and Semidefinite Programming

Robust Second-Order Nonconvex Optimization and Its Application to Low Rank Matrix Sensing

Robust Low-Rank Subspace Segmentation with Semidefinite Guarantees

Learning Deterministic Surrogates for Robust Convex QCQPs

A Fast Saddle-Point Dynamical System Approach to Robust Deep Learning

Robust optimization for adversarial learning with finite sample complexity guarantees

SOS-convex Semi-algebraic Programs and its Applications to Robust Optimization: A Tractable Class of Nonsmooth Convex Optimization

A Stochastic Subgradient Method for Distributionally Robust Non-Convex Learning

Decomposed Structured Subsets for Semidefinite Optimization

Robust Learning with Imperfect Privileged Information.

SDPs and Robust Satisfiability of Promise CSP

A feasible method for general convex low-rank SDP problems

Distributionally Robust Learning with Weakly Convex Losses: Convergence Rates and Finite-Sample Guarantees

Local Competition and Uncertainty for Adversarial Robustness in Deep Learning

Enhancing low-rank solutions in semidefinite relaxations of Boolean quadratic problems

SDP Duals without Duality Gaps for a Class of Convex Minimax Programs