Abstract:We consider the problem of signal estimation in a generalized linear model (GLM). GLMs include many canonical problems in statistical estimation, such as linear regression, phase retrieval, and 1-bit compressed sensing. Recent work has precisely characterized the asymptotic minimum mean-squared error (MMSE) for GLMs with i.i.d. Gaussian sensing matrices. However, in many models there is a significant gap between the MMSE and the performance of the best known feasible estimators. In this work, we address this issue by considering GLMs defined via spatially coupled sensing matrices. We propose an efficient approximate message passing (AMP) algorithm for estimation and prove that with a simple choice of spatially coupled design, the MSE of a carefully tuned AMP estimator approaches the asymptotic MMSE in the high-dimensional limit. To prove the result, we first rigorously characterize the asymptotic performance of AMP for a GLM with a generic spatially coupled design. This characterization is in terms of a deterministic recursion (`state evolution') that depends on the parameters defining the spatial coupling. Then, using a simple spatially coupled design and a judicious choice of functions for the AMP algorithm, we analyze the fixed points of the resulting state evolution and show that it achieves the asymptotic MMSE. Numerical results for phase retrieval and rectified linear regression show that spatially coupled designs can yield substantially lower MSE than i.i.d. Gaussian designs at finite dimensions when used with AMP algorithms.

Orthogonal Approximate Message-Passing for Spatially Coupled Linear Models

A Unitary Transform Based Generalized Approximate Message Passing

Unitary Approximate Message Passing for Sparse Bayesian Learning

Concise Derivation of Complex Bayesian Approximate Message Passing Via Expectation Propagation.

Long-Memory Message-Passing for Spatially Coupled Systems

Estimation of Sparse Massive MIMO-OFDM Channels With Approximately Common Support.

An Expectation Propagation Perspective on Approximate Message Passing

On the Convergence of Orthogonal/Vector AMP: Long-Memory Message-Passing Strategy

On OAMP: Impact of the Orthogonal Principle

Convolutional Approximate Message-Passing

Orthogonal AMP for Problems with Multiple Measurement Vectors and/or Multiple Transforms

Approximate Message Passing for orthogonally invariant ensembles: Multivariate non-linearities and spectral initialization

Nonuniform Support Recovery from Noisy Random Measurements by Orthogonal Matching Pursuit.

A Concise Tutorial on Approximate Message Passing

Phase Retrieval from Quantized Measurements Via Approximate Message Passing

Bayes-Optimal Estimation in Generalized Linear Models via Spatial Coupling

Linear Operator Approximate Message Passing (OpAMP)

Approximate Message Passing for the Matrix Tensor Product Model

Unifying AMP Algorithms for Rotationally-Invariant Models