Abstract:Temporal graph is an abstraction for modeling dynamic systems that consist of evolving interaction elements. In this paper, we aim to solve an important yet neglected problem -- how to learn information from high-order neighbors in temporal graphs? -- to enhance the informativeness and discriminativeness for the learned node representations. We argue that when learning high-order information from temporal graphs, we encounter two challenges, i.e., computational inefficiency and over-smoothing, that cannot be solved by conventional techniques applied on static graphs. To remedy these deficiencies, we propose a temporal propagation-based graph neural network, namely TPGNN. To be specific, the model consists of two distinct components, i.e., propagator and node-wise encoder. The propagator is leveraged to propagate messages from the anchor node to its temporal neighbors within $k$-hop, and then simultaneously update the state of neighborhoods, which enables efficient computation, especially for a deep model. In addition, to prevent over-smoothing, the model compels the messages from $n$-hop neighbors to update the $n$-hop memory vector preserved on the anchor. The node-wise encoder adopts transformer architecture to learn node representations by explicitly learning the importance of memory vectors preserved on the node itself, that is, implicitly modeling the importance of messages from neighbors at different layers, thus mitigating the over-smoothing. Since the encoding process will not query temporal neighbors, we can dramatically save time consumption in inference. Extensive experiments on temporal link prediction and node classification demonstrate the superiority of TPGNN over state-of-the-art baselines in efficiency and robustness.

Learning multi‐modal recurrent neural networks with target propagation

Deep Spiking Neural Network Using Spatio-temporal Backpropagation with Variable Resistance.

The Balanced Multi-Modal Spiking Neural Networks with Online Loss Adjustment and Time Alignment

Multi-Target Regression Via Robust Low-Rank Learning.

Extraction Propagation

Hierarchical Conflict Propagation: Sequence Learning in a Recurrent Deep Neural Network

Learning multiple layers of representation

Multi-target regression via stochastic configuration networks with modular stacked structure

TPGNN: Learning High-order Information in Dynamic Graphs via Temporal Propagation

Dual Propagation: Accelerating Contrastive Hebbian Learning with Dyadic Neurons

Online Multi-Target Tracking Using Recurrent Neural Networks

Temporal dendritic heterogeneity incorporated with spiking neural networks for learning multi-timescale dynamics

Performing Multi-Target Regression via a Parameter Sharing-Based Deep Network

Transfer learning for temporal nodes Bayesian networks

Supervised Learning in Multilayer Spiking Neural Networks With Spike Temporal Error Backpropagation

Backpropagation through time learning for recurrence-aware long-term cognitive networks

Learning How to Propagate Messages in Graph Neural Networks

Neural Graph Machines: Learning Neural Networks Using Graphs

Gradient-adjusted Incremental Target Propagation Provides Effective Credit Assignment in Deep Neural Networks

Tree Memory Networks for Modelling Long-term Temporal Dependencies

Propagation Model Search for Graph Neural Networks.