Abstract:Approximate Message Passing (AMP) is an efficient iterative parameter-estimation technique for certain high-dimensional linear systems with non-Gaussian distributions, such as sparse systems. In AMP, a so-called Onsager term is added to keep estimation errors approximately Gaussian. Orthogonal AMP (OAMP) does not require this Onsager term, relying instead on an orthogonalization procedure to keep the current errors uncorrelated with (i.e., orthogonal to) past errors. In this paper, we show the generality and significance of the orthogonality in ensuring that errors are “asymptotically independently and identically distributed Gaussian” (AIIDG). This AIIDG property, which is essential for the attractive performance of OAMP, holds for separable functions. We present a simple and versatile procedure to establish the orthogonality through Gram-Schmidt (GS) orthogonalization, which is applicable to any prototype. We show that different AMP-type algorithms, such as expectation propagation (EP), turbo, AMP and OAMP, can be unified under the orthogonal principle. The simplicity and generality of OAMP provide efficient solutions for estimation problems beyond the classical linear models. As an example, we study the optimization of OAMP via the GS model and GS orthogonalization. More related applications will be discussed in a companion paper where new algorithms are developed for problems with multiple constraints and multiple measurement variables.

Augmented Infinitesimal Perturbation Analysis: an Alternate Explanation

Aggregation of simulation and infinitesimal perturbation analysis in discrete event dynamic system

Bias Properties Of Infinitesimal Perturbation Analysis For Systems With Parallel Servers

Consistency of infinitesimal perturbation analysis estimators with rates

Strongly Consistent Steady-State Derivative Estimates

Performance Regulation of Event-Driven Dynamical Systems Using Infinitesimal Perturbation Analysis

Strong Consistency Of Sample Path Derivative Estimates

Bias Properties Of Infinitesimal Perturbation Analysis For Multiserver Queues

A New Unbiased Stochastic Derivative Estimator for Discontinuous Sample Performances with Structural Parameters.

Variance Property of Discontinuous Perturbation Analysis

An Infinitesimal Perturbation Analysis Algorithm For A Multiclass G/G/1 Queue

Smale's α-theory for inexact Newton methods under the γ-condition

Convexity of sample path performance and strong consistency of infinitesimal perturbation analysis estimates

Extensions and Generalizations of Smoothed Perturbation Analysis in a Generalized Semi-Markov Process Framework

Steady-state Sample Path Derivative Estimators

On Comparison of Steady-State Infinitesimal Perturbation Analysis and Likelihood Ratio Derivative Estimates.

Discontinuous Perturbation Analysis of Discrete-Event Dynamic Systems

On OAMP: Impact of the Orthogonal Principle

Infinitesimal Perturbation Analysis for Personalized Cancer Therapy Design

IPA in the Loop: Control Design for Throughput Regulation in Computer Processors

A Non-asymptotic Analysis of Generalized Approximate Message Passing Algorithms with Right Rotationally Invariant Designs