Abstract:Temporal graph is an abstraction for modeling dynamic systems that consist of evolving interaction elements. In this paper, we aim to solve an important yet neglected problem -- how to learn information from high-order neighbors in temporal graphs? -- to enhance the informativeness and discriminativeness for the learned node representations. We argue that when learning high-order information from temporal graphs, we encounter two challenges, i.e., computational inefficiency and over-smoothing, that cannot be solved by conventional techniques applied on static graphs. To remedy these deficiencies, we propose a temporal propagation-based graph neural network, namely TPGNN. To be specific, the model consists of two distinct components, i.e., propagator and node-wise encoder. The propagator is leveraged to propagate messages from the anchor node to its temporal neighbors within $k$-hop, and then simultaneously update the state of neighborhoods, which enables efficient computation, especially for a deep model. In addition, to prevent over-smoothing, the model compels the messages from $n$-hop neighbors to update the $n$-hop memory vector preserved on the anchor. The node-wise encoder adopts transformer architecture to learn node representations by explicitly learning the importance of memory vectors preserved on the node itself, that is, implicitly modeling the importance of messages from neighbors at different layers, thus mitigating the over-smoothing. Since the encoding process will not query temporal neighbors, we can dramatically save time consumption in inference. Extensive experiments on temporal link prediction and node classification demonstrate the superiority of TPGNN over state-of-the-art baselines in efficiency and robustness.

A Neural Network Empowered Belief Propagation Algorithm Optimized for Short-Cycles in Tanner Graph

New Low Complexity Belief Propagation Decoding of Low Density Parity Check Codes

XJ-BP: Express Journey Belief Propagation Decoding for Polar Codes

Evolving Neural Networks Using the Hybrid of Ant Colony Optimization and Bp Algorithms

A Simple, Fast Diverse Decoding Algorithm for Neural Generation

Belief Propagation Decoding for Short-Length Codes Based on Sparse Tanner Graph

Deep Attentive Belief Propagation: Integrating Reasoning and Learning for Solving Constraint Optimization Problems

Deep learning via message passing algorithms based on belief propagation

A Scalable Graph Neural Network Decoder for Short Block Codes

Belief Propagation with Gradual Edge Removal for Raptor Codes over AWGN Channel

Belief propagation for general graphical models with loops

Iteration-Aware Dynamic Quantization Neural Belief Propagation Decoder

A Model-Driven Quasi-ResNet Belief Propagation Neural Network Decoder for LDPC Codes.

Communication-Efficient Parallel Belief Propagation for Latent Dirichlet Allocation

TPGNN: Learning High-order Information in Dynamic Graphs via Temporal Propagation

Neural Enhanced Belief Propagation on Factor Graphs

Spiking Neural Belief Propagation Decoder for Short Block Length LDPC Codes

Belief Propagation by Message Passing in Junction Trees: Computing Each Message Faster Using GPU Parallelization

A self-adaptive hierarchical belief propagation algorithm

Biologically Plausible Training of Deep Neural Networks Using a Top-down Credit Assignment Network

Nonparametric generalized belief propagation based on pseudo-junction tree for cooperative localization in wireless networks