Uncovering Selective State Space Model's Capabilities in Lifelong Sequential Recommendation

Jiyuan Yang,Yuanzi Li,Jingyu Zhao,Hanbing Wang,Muyang Ma,Jun Ma,Zhaochun Ren,Mengqi Zhang,Xin Xin,Zhumin Chen,Pengjie Ren
2024-03-25
Abstract:Sequential Recommenders have been widely applied in various online services, aiming to model users' dynamic interests from their sequential interactions. With users increasingly engaging with online platforms, vast amounts of lifelong user behavioral sequences have been generated. However, existing sequential recommender models often struggle to handle such lifelong sequences. The primary challenges stem from computational complexity and the ability to capture long-range dependencies within the sequence. Recently, a state space model featuring a selective mechanism (i.e., Mamba) has emerged. In this work, we investigate the performance of Mamba for lifelong sequential recommendation (i.e., length>=2k). More specifically, we leverage the Mamba block to model lifelong user sequences selectively. We conduct extensive experiments to evaluate the performance of representative sequential recommendation models in the setting of lifelong sequences. Experiments on two real-world datasets demonstrate the superiority of Mamba. We found that RecMamba achieves performance comparable to the representative model while significantly reducing training duration by approximately 70% and memory costs by 80%. Codes and data are available at \url{
Computer Science
What problem does this paper attempt to address?
The paper attempts to address the problem of computational complexity and difficulty in capturing long-distance dependencies encountered by existing models when dealing with long sequences in long-term user behavior sequence recommendations. Specifically, as users interact more and more on online platforms, a large amount of long-term user behavior sequence data is generated. However, existing sequential recommendation models often struggle to effectively handle these long-term sequences, primarily due to high computational complexity and difficulty in capturing long-distance dependencies within the sequences. To address these issues, the paper introduces a new state space model (i.e., Mamba) with a selection mechanism that can effectively handle long-term user behavior sequences. The paper evaluates Mamba's performance in long-term sequence recommendations through experiments and compares it with existing representative recommendation models. The experimental results show that Mamba not only outperforms other models in terms of performance but also has significant advantages in training time and memory consumption.