Abstract:Approximate Message Passing (AMP), originally designed to solve high-dimensional linear inverse problems, has found broad applications in signal processing and statistical inference. Among its key variants, Vector Approximate Message Passing (VAMP) and Generalized Approximate Survey Propagation (GASP) have demonstrated effectiveness even in scenarios where the assumed generative models differ from the true models. However, the maximum a posteriori (MAP) versions of VAMP and GASP have limitations: VAMP is restricted to differentiable priors and likelihoods, while GASP requires the measurement matrix to have independent identically distributed (i.i.d.) elements. To overcome these limitations, this paper introduces a new algorithm, Vector Approximate Survey Propagation (VASP). VASP utilizes survey propagation to handle non-differentiable priors and likelihoods, along with employs vector-form messages to account for correlations in the measurement matrix. Simulations reveal that VASP significantly surpasses VAMP and GASP in estimation accuracy, particularly when the assumed prior is discrete-supported and the measurement matrix is non-i.i.d.. Additionally, the state evolution (SE) of VASP, derived heuristically, accurately reflects the per-iteration mean squared error (MSE). A comparison between the SE and the free energy computed by Takahashi and Kabashima under the one-step replica symmetry breaking (1RSB) ansatz shows that the SE's fixed-point equations align with the free energy's saddle point equations. This suggests that VASP efficiently implements the postulated MAP estimator (which is NP-hard in the worst case) with cubic computational complexity, assuming the 1RSB ansatz is valid.

Unifying Message Passing Algorithms under the Framework of Constrained Bethe Free Energy Minimization

An Expectation Propagation Perspective on Approximate Message Passing

Concise Derivation of Complex Bayesian Approximate Message Passing Via Expectation Propagation.

Unitary Approximate Message Passing for Sparse Bayesian Learning

A Unitary Transform Based Generalized Approximate Message Passing

A Unified Bayesian Inference Framework for Generalized Linear Models

Message Passing with L1 Penalized KL Minimization

A unifying tutorial on Approximate Message Passing

A Concise Tutorial on Approximate Message Passing

Distributed Inference With Variational Message Passing in Gaussian Graphical Models: Tradeoffs in Message Schedules and Convergence Conditions

Convergent Message-Passing Algorithms for Inference over General Graphs with Convex Free Energies

A New Insight into GAMP and AMP.

Neuronal message passing using Mean-field, Bethe, and Marginal approximations

Concise Derivation for Generalized Approximate Message Passing Using Expectation Propagation

Hybrid Vector Message Passing for Generalized Bilinear Factorization

Convergence of Some Convex Message Passing Algorithms to a Fixed Point

Sparse Vector Recovery: Bernoulli-Gaussian Message Passing

Vector Approximate Survey Propagation

Approximate Message Passing for sparse matrices with application to the equilibria of large ecological Lotka-Volterra systems

Convex Combination Belief Propagation Algorithms

Understanding belief propagation and its generalizations