Abstract:The problem of recovering a signal $\boldsymbol x\in \mathbb{R}^n$ from a quadratic system $\{y_i=\boldsymbol x^\top\boldsymbol A_i\boldsymbol x,\ i=1,\ldots,m\}$ with full-rank matrices $\boldsymbol A_i$ frequently arises in applications such as unassigned distance geometry and sub-wavelength imaging. With i.i.d. standard Gaussian matrices $\boldsymbol A_i$, this paper addresses the high-dimensional case where $m\ll n$ by incorporating prior knowledge of $\boldsymbol x$. First, we consider a $k$-sparse $\boldsymbol x$ and introduce the thresholded Wirtinger flow (TWF) algorithm that does not require the sparsity level $k$. TWF comprises two steps: the spectral initialization that identifies a point sufficiently close to $\boldsymbol x$ (up to a sign flip) when $m=O(k^2\log n)$, and the thresholded gradient descent which, when provided a good initialization, produces a sequence linearly converging to $\boldsymbol x$ with $m=O(k\log n)$ measurements. Second, we explore the generative prior, assuming that $x$ lies in the range of an $L$-Lipschitz continuous generative model with $k$-dimensional inputs in an $\ell_2$-ball of radius $r$. With an estimate correlated with the signal, we develop the projected gradient descent (PGD) algorithm that also comprises two steps: the projected power method that provides an initial vector with $O\big(\sqrt{\frac{k \log L}{m}}\big)$ $\ell_2$-error given $m=O(k\log(Lnr))$ measurements, and the projected gradient descent that refines the $\ell_2$-error to $O(\delta)$ at a geometric rate when $m=O(k\log\frac{Lrn}{\delta^2})$. Experimental results corroborate our theoretical findings and show that: (i) our approach for the sparse case notably outperforms the existing provable algorithm sparse power factorization; (ii) leveraging the generative prior allows for precise image recovery in the MNIST dataset from a small number of quadratic measurements.

A gradient based technique for generating sparse representation in function approximation

Sparse multiresolution representations with adaptive kernels

Sparse Least Squares Support Vector Machine for Function Estimation

Greedy Sparsity-Constrained Optimization

Sparse Gaussian processes for solving nonlinear PDEs

A type-II maximum-likelihood approach to Gaussian scale mixture-based sparse regression Kriging

Dimension Reduction via Gaussian Ridge Functions

Non-uniform Nyström approximation for sparse kernel regression: Theoretical analysis and experimental evaluation

Surface Approximation Via Sparse Representation and Parameterization Optimization

Differentially Private Optimization with Sparse Gradients

An Efficient Two-Stage Sparse Representation Method.

A Constructive Approach to High-dimensional Regression

Estimation of Linear Functionals in High-Dimensional Linear Models: From Sparsity to Nonsparsity

Solving Quadratic Systems with Full-Rank Matrices Using Sparse or Generative Priors

A Semismooth Newton Algorithm for High-Dimensional Nonconvex Sparse Learning

Learning-Based Low-Rank Approximations

Approximation of RKHS Functionals by Neural Networks

Slowly Varying Regression under Sparsity

Sparse polynomial approximations for affine parametric saddle point problems

Newton Greedy Pursuit: A Quadratic Approximation Method for Sparsity-Constrained Optimization.

An efficient active-set method with applications to sparse approximations and risk minimization