HSA: Hyperbolic Self-attention for Sequential Recommendation

Peizhong Hou,Haiyang Wang,Tianming Li,Junchi Yan

DOI: https://doi.org/10.1007/978-981-97-2387-4_17

2024-01-01

Abstract:Recently, researchers apply various deep neural networks to the task of sequential recommendation, which captures dynamics of user preference from user behavior data to make accurate recommendation. Self-attention based approaches have been proposed to effectively identify relevant items and better capture long-term dependencies, achieving competitive results in sequential recommendation domain. However, most existing methods perform in the Euclidean space, which expands only polynomially, limiting the capacity of models. Besides, methods typically do not consider and leverage latent hierarchical structures existing in real-world datasets. To this end, we propose to learn representations in hyperbolic space for sequential recommendation, bringing two advantages. First, hyperbolic space expands exponentially and thus provides higher representation ability. Second, it is able to effectively model the latent hierarchical structures, which are indicated by the power-law distributions of user behavior sequences. Specifically, we propose a novel hyperbolic self-attention model, which learns item embeddings in hyperbolic space and adopts self-attention to model user sequence representation, using hyperbolic distance to measure preference and make recommendation. Extensive experiments conducted on three real-world datasets demonstrate the superiority of our proposed hyperbolic embedding approach over various competitive baselines, including Euclidean self-attention counterpart. We apply the proposed hyperbolic embedding method to classic sequential recommendation models and observe improvement, showing it a general technique which can boost other models.