Abstract:Belief Propagation (BP) is an important message-passing algorithm for various reasoning tasks over graphical models, including solving the Constraint Optimization Problems (COPs). It has been shown that BP can achieve state-of-the-art performance on various benchmarks by mixing old and new messages before sending the new one, i.e., damping. However, existing methods of tuning a static damping factor for BP not only are laborious but also harm their performance. Moreover, existing BP algorithms treat each variable node's neighbors equally when composing a new message, which also limits their exploration ability. To address these issues, we seamlessly integrate BP, Gated Recurrent Units (GRUs), and Graph Attention Networks (GATs) within the message-passing framework to reason about dynamic weights and damping factors for composing new BP messages. Our model, Deep Attentive Belief Propagation (DABP), takes the factor graph and the BP messages in each iteration as the input and infers the optimal weights and damping factors through GRUs and GATs, followed by a multi-head attention layer. Furthermore, unlike existing neural-based BP variants, we propose a novel self-supervised learning algorithm for DABP with a smoothed solution cost, which does not require expensive training labels and also avoids the common out-of-distribution issue through efficient online learning. Extensive experiments show that our model significantly outperforms state-of-the-art baselines.

Recursive Belief Propagation --- Belief Propagation in a Well-order

An Expectation Propagation Perspective on Approximate Message Passing

Circular Belief Propagation for Approximate Probabilistic Inference

New Low Complexity Belief Propagation Decoding of Low Density Parity Check Codes

A Simple Insight into Iterative Belief Propagation's Success

A self-adaptive hierarchical belief propagation algorithm

XJ-BP: Express Journey Belief Propagation Decoding for Polar Codes

Understanding belief propagation and its generalizations

A Neural Network Empowered Belief Propagation Algorithm Optimized for Short-Cycles in Tanner Graph

Deep Attentive Belief Propagation: Integrating Reasoning and Learning for Solving Constraint Optimization Problems

Convex Combination Belief Propagation Algorithms

Deep learning via message passing algorithms based on belief propagation

Belief Propagation with Gradual Edge Removal for Raptor Codes over AWGN Channel

Check Belief Propagation Decoding of LDPC Codes

Iterative Belief Propagation for Sparse Combinatorial Optimization

Belief propagation algorithm for computing correlation functions in finite-temperature quantum many-body systems on loopy graphs

Convergence of Generalized Belief Propagation Algorithm on Graphs with Motifs

Communication-Efficient Parallel Belief Propagation for Latent Dirichlet Allocation

Belief propagation for general graphical models with loops

Fixing the Bethe approximation: How structural modifications in a graph improve belief propagation

Nonparametric generalized belief propagation based on pseudo-junction tree for cooperative localization in wireless networks