Abstract:In signal processing and wireless communications, the generalized linear system (GLS) has been widely used to evaluate the impact of nonlinear preprocessing on receiver performance. Although generalized approximation message passing (GAMP) is a state-of-the-art algorithm for GLS signal recovery, it is only applicable to measurement matrices with independent and identically distributed (IID) elements. To overcome this limitation, the generalized orthogonal/vector approximate message passing (GOAMP/GVAMP) algorithm for unitarily invariant measurement matrices was developed and shown to be replica Bayes optimal in uncoded GLS. However, due to its complicated state evolution (SE), the information-theoretic limit of GOAMP/GVAMP remains an open problem for arbitrary input distributions. In this paper, we provide a rigorous achievable rate analysis of GOAMP/GVAMP in GLS, establishing its information-theoretic limit (i.e., maximum achievable rate). Specifically, GOAMP/GVAMP involves a dual-input-dual-output linear detector (LD) paired with two nonlinear detectors (NLD), making the achievable rate difficult to analyze. To address this issue, we convert the fully unfolded state evolution (SE) of a multi-layer iterative algorithm into an equivalent single-input single-output variational SE (VSE). The achievable rate of GOAMP/GVAMP is calculated using the VSE and the mutual information and minimum mean square error (I-MMSE) lemma. Also, the optimal coding principle for maximizing the achievable rate is provided. Furthermore, the methods presented in the paper can be easily extended to analyze the achievable rate and optimal code design for high-dimensional neural networks using multi-layer AMP-type algorithms. Clipping and quantization, two common nonlinear functions, are used as examples to show the advantages of GOAMP/GVAMP over existing techniques, such as the maximum ratio combining receivers based on the linearized model and the cascading GOAMP/GVAMP with separate LD and NLD. In numerical results, a kind of low-density parity-check (LDPC) code is optimized for GOAMP/GVAMP to approach the theoretical limit within dB. The finite-length performances of the GOAMP/GVAMP with optimized LDPC codes are within dB away from the theoretical limits in GLS.

Generalized Approximate Survey Propagation for High-Dimensional Estimation

A Unitary Transform Based Generalized Approximate Message Passing

An AMP-Based Low Complexity Generalized Sparse Bayesian Learning Algorithm

Unitary Approximate Message Passing for Sparse Bayesian Learning

Concise Derivation of Complex Bayesian Approximate Message Passing Via Expectation Propagation.

Bilinear Adaptive Generalized Vector Approximate Message Passing

A Unified Bayesian Inference Framework for Generalized Linear Models

An Expectation Propagation Perspective on Approximate Message Passing

Vector Approximate Survey Propagation

A New Insight into GAMP and AMP.

Bayes-Optimal Estimation in Generalized Linear Models via Spatial Coupling

Computationally Efficient Sparse Bayesian Learning Via Generalized Approximate Message Passing

Generalized Memory Approximate Message Passing for Generalized Linear Model.

Generalized Approximate Message-Passing for Compressed Sensing with Sublinear Sparsity

Achievable Rates of Generalized Linear Systems With Orthogonal/Vector AMP Receiver

Sparse Estimation Based on a New Random Regularized Matching Pursuit Generalized Approximate Message Passing Algorithm.

Approximate Survey Propagation for Statistical Inference

Discrete-Valued Signal Estimation via Low-Complexity Message Passing Algorithm for Highly Correlated Measurements

K-step Vector Approximate Survey Propagation

Concise Derivation for Generalized Approximate Message Passing Using Expectation Propagation