Abstract:Research into methods for reasoning under uncertainty is currently one of the most exciting areas of artificial intelligence, largely because it has recently become possible to record, store, and process large amounts of data. While impressive achievements have been made in pattern classification problems such as handwritten character recognition, face detection, speaker identification, and prediction of gene function, it is even more exciting that researchers are on the verge of introducing systems that can perform large-scale combinatorial analyses of data, decomposing the data into interacting components. For example, computational methods for automatic scene analysis are now emerging in the computer vision community. These methods decompose an input image into its constituent objects, lighting conditions, motion patterns, etc. Two of the main challenges are finding effective representations and models in specific applications and finding efficient algorithms for inference and learning in these models. In this paper, we advocate the use of graph-based probability models and their associated inference and learning algorithms. We review exact techniques and various approximate, computationally efficient techniques, including iterated conditional modes, the expectation maximization (EM) algorithm, Gibbs sampling, the mean field method, variational techniques, structured variational techniques and the sum-product algorithm ("loopy" belief propagation). We describe how each technique can be applied in a vision model of multiple, occluding objects and contrast the behaviors and performances of the techniques using a unifying cost function, free energy.

A visual introduction to Gaussian Belief Propagation

Learning Gaussian Networks

Understanding belief propagation and its generalizations

Learning in Deep Factor Graphs with Gaussian Belief Propagation

Gaussian Ensemble Belief Propagation for Efficient Inference in High-Dimensional Systems

Improved Generalized Belief Propagation for Vision Processing

Distributed Inference over Linear Models using Alternating Gaussian Belief Propagation

Circular Belief Propagation for Approximate Probabilistic Inference

A comparison of algorithms for inference and learning in probabilistic graphical models

Convergence Analysis of Gaussian SPAWN under High-Order Graphical Models.

Composite Inference for Gaussian Processes

GaussianProcesses.jl: A Nonparametric Bayes package for the Julia Language

Convex Combination Belief Propagation Algorithms

Learning General Latent-Variable Graphical Models with Predictive Belief Propagation

Variable Sigma Gaussian Processes: an Expectation Propagation Perspective

Learning unbelievable probabilities

Approximate Bayesian Image Interpretation using Generative Probabilistic Graphics Programs

Neural Enhanced Belief Propagation on Factor Graphs

Generalized Approximate Survey Propagation for High-Dimensional Estimation

Learning unbelievable marginal probabilities

The Poisson Gamma Belief Network