Cascade modeling with multihead self-attention

Chaochao Liu,Wenjun Wang,Pengfei Jiao,Xue Chen,Yueheng Sun
DOI: https://doi.org/10.1109/ijcnn48605.2020.9207418
2020-01-01
Abstract:Modeling how information diffuses across social network platforms can be widely used. Recently, researchers have used deep learning methods to model information cascades and forecast their progression without dependence on the hypothesis of the underlying diffusion model. Most of these studies use sequential models (e.g., recurrent neural networks, RNNs) and model cascades of information spread without using the network structure information. However, the network structure information substantially affects information spread, cross-dependence should be considered in cascade modeling, and recurrent neural networks produce poor results on long sequence modeling. To solve these issues, in this paper, we present a new cascade modeling method with the multihead self-attention mechanism. We design an encoder that combines network structure information with multihead self-attention to learn the representations of cascades and consider diverse user dependencies on the network. Experiments are conducted on both synthetic and real-world datasets. The results show that the proposed method performs better on the long sequence cascade prediction than the state-of-the-art methods.
What problem does this paper attempt to address?