Abstract:Factor graphs are important models for succinctly representing probability distributions in machine learning, coding theory, and statistical physics. Several computational problems, such as computing marginals and partition functions, arise naturally when working with factor graphs. Belief propagation is a widely deployed iterative method for solving these problems. However, despite its significant empirical success, several questions regarding the correctness and efficiency of belief propagation remain open. The Bethe approximation is an optimization-based method for approximating the partition functions. While it is known that the stationary points of the Bethe approximation coincide with the fixed points of belief propagation, in general, the relation between the Bethe approximation and the partition function is not well understood. It has been observed that for a few classes of factor graphs, the Bethe approximation gives a lower bound to the partition function, which distinguishes them from the general case, where neither a lower bound nor an upper bound holds universally. This has been rigorously proved for permanents and for attractive graphical models. Here, we consider bipartite factor graphs over binary alphabet and show that if the local constraints satisfy a certain analytic property, the Bethe approximation is a lower bound to the partition function, generalizing an analogous inequality between the permanent and the Bethe permanent of a matrix with non-negative entries. We arrive at this result by viewing the factor graphs through the lens of polynomials, which allows us to reformulate the Bethe approximation as an optimization problem involving polynomials. The sufficient condition for our lower bound property to hold is inspired by the recent developments in the theory of real stable polynomials. We believe that this way of viewing factor graphs and its connection to real stability might lead to a better understanding of belief propagation and factor graphs in general.

On the uniqueness of loopy belief propagation fixed points

Convergence of MCMC and Loopy BP in the Tree Uniqueness Region for the Hard-Core Model

Local Max-Entropy and Free Energy Principles, Belief Diffusions and their Singularities

Emergent infinite-randomness fixed points from the extensive random bipartitions of the spin-1 Affleck-Kennedy-Lieb-Tasaki topological state

Approximate Inference and Constrained Optimization

A Homological Approach to Belief Propagation and Bethe Approximations

Fixing the Bethe approximation: How structural modifications in a graph improve belief propagation

Zero-Temperature Limit of a Convergent Algorithm to Minimize the Bethe Free Energy

Learning unbelievable probabilities

Belief propagation algorithm for computing correlation functions in finite-temperature quantum many-body systems on loopy graphs

Graph polynomials and approximation of partition functions with Loopy Belief Propagation

Loopy annealing belief propagation for vertex cover and matching: convergence, LP relaxation, correctness and Bethe approximation

Convergence of Generalized Belief Propagation Algorithm on Graphs with Motifs

Learning unbelievable marginal probabilities

An improved Belief Propagation algorithm finds many Bethe states in the random field Ising model on random graphs

Belief Propagation, Bethe Approximation and Polynomials

Ising Model on Locally Tree-like Graphs: Uniqueness of Solutions to Cavity Equations

Uniqueness of BP fixed point for the Potts model and applications to community detection

Positive Fixed Points for Nonlinear Operators

Percolation at the uniqueness threshold via subgroup relativization

The Bethe Partition Function of Log-supermodular Graphical Models