Abstract:We often encounter probability distributions given as unnormalized products of non-negative functions. The factorization structures are represented by hypergraphs called factor graphs. Such distributions appear in various fields, including statistics, artificial intelligence, statistical physics, error correcting codes, etc. Given such a distribution, computations of marginal distributions and the normalization constant are often required. However, they are computationally intractable because of their computational costs. One successful approximation method is Loopy Belief Propagation (LBP) algorithm. The focus of this thesis is an analysis of the LBP algorithm. If the factor graph is a tree, i.e. having no cycle, the algorithm gives the exact quantities. If the factor graph has cycles, however, the LBP algorithm does not give exact results and possibly exhibits oscillatory and non-convergent behaviors. The thematic question of this thesis is "How the behaviors of the LBP algorithm are affected by the discrete geometry of the factor graph?" The primary contribution of this thesis is the discovery of a formula that establishes the relation between the LBP, the Bethe free energy and the graph zeta function. This formula provides new techniques for analysis of the LBP algorithm, connecting properties of the graph and of the LBP and the Bethe free energy. We demonstrate applications of the techniques to several problems including (non) convexity of the Bethe free energy, the uniqueness and stability of the LBP fixed point. We also discuss the loop series initiated by Chertkov and Chernyak. The loop series is a subgraph expansion of the normalization constant, or partition function, and reflects the graph geometry. We investigate theoretical natures of the series. Moreover, we show a partial connection between the loop series and the graph zeta function.

Local Message Passing on Frustrated Systems

An Expectation Propagation Perspective on Approximate Message Passing

Structural Optimization of Factor Graphs for Symbol Detection via Continuous Clustering and Machine Learning

Tensor Network Message Passing

A residual-based message passing algorithm for constraint satisfaction problems

Dynamic tree block coordinate ascent

Message passing all the way up

Discrete geometric analysis of message passing algorithm on graphs

Convergence of Some Convex Message Passing Algorithms to a Fixed Point

Gaussian Message Passing for Cooperative Localization in Wireless Networks.

Evaluating Large Graph Processing in MapReduce Based on Message Passing

Factor Graph and Damped Expectation Propagation Based Passive Localization

Distributed Cooperative Localization Based on Gaussian Message Passing on Factor Graph in Wireless Networks

Overcoming the complexity barrier of the dynamic message-passing method in networks with fat-tailed degree distributions

Gaussian Message Passing on Linear Models: an Update

Weighted Community Detection and Data Clustering Using Message Passing

Optimality of Message-Passing Architectures for Sparse Graphs

A New Approach to Laplacian Solvers and Flow Problems

Factor graphs and the sum-product algorithm

MS2MP: A Min-Sum Message Passing Algorithm for Motion Planning