Abstract:In applications such as e-commerce, online education, and streaming services, sequential recommendation systems play a critical role. Despite the excellent performance of self-attention-based sequential recommendation models in capturing dependencies between items in user interaction history, their quadratic complexity and lack of structural bias limit their applicability. Recently, some works have replaced the self-attention module in sequential recommenders with Mamba, which has linear complexity and structural bias. However, these works have not noted the complementarity between the two approaches. To address this issue, this paper proposes a new hybrid recommendation framework, Mamba combined with Low-Rank decomposed Self-Attention for Sequential Recommendation (MLSA4Rec), whose complexity is linear with respect to the length of the user's historical interaction sequence. Specifically, MLSA4Rec designs an efficient Mamba-LSA interaction module. This module introduces a low-rank decomposed self-attention (LSA) module with linear complexity and injects structural bias into it through Mamba. The LSA module analyzes user preferences from a different perspective and dynamically guides Mamba to focus on important information in user historical interactions through a gated information transmission mechanism. Finally, MLSA4Rec combines user preference information refined by the Mamba and LSA modules to accurately predict the user's next possible interaction. To our knowledge, this is the first study to combine Mamba and self-attention in sequential recommendation systems. Experimental results show that MLSA4Rec outperforms existing self-attention and Mamba-based sequential recommendation models in recommendation accuracy on three real-world datasets, demonstrating the great potential of Mamba and self-attention working together.

MAE4Rec

MAE4Rec: Storage-saving Transformer for Sequential Recommendations

RaSeRec: Retrieval-Augmented Sequential Recommendation

EASRec: External Attentive Efficient Sequential Recommender

EASRec: Elastic Architecture Search for Efficient Long-term Sequential Recommender Systems

Mamba4Rec: Towards Efficient Sequential Recommendation with Selective State Space Models

SSE4Rec: Sequential recommendation with subsequence extraction

AdaRec: Adaptive Sequential Recommendation for Reinforcing Long-term User Engagement

Ada-Retrieval: An Adaptive Multi-Round Retrieval Paradigm for Sequential Recommendations

SSD4Rec: A Structured State Space Duality Model for Efficient Sequential Recommendation

An Empirical Study of Training ID-Agnostic Multi-modal Sequential Recommenders

STRec: Sparse Transformer for Sequential Recommendations.

Modeling and predicting user preferences with multiple item attributes for sequential recommendations

Beyond Inter-Item Relations: Dynamic Adaption for Enhancing LLM-Based Sequential Recommendation

Recognizing Semantics-Consistent Subsequences for Sequential Recommendation

EchoMamba4Rec: Harmonizing Bidirectional State Space Models with Spectral Filtering for Advanced Sequential Recommendation

CSRec: Rethinking Sequential Recommendation from A Causal Perspective

MLSA4Rec: Mamba Combined with Low-Rank Decomposed Self-Attention for Sequential Recommendation

Time Interval Aware Self-Attention for Sequential Recommendation

Enhancing Repeat-Aware Recommendation from a Temporal-Sequential Perspective

A General Tail Item Representation Enhancement Framework for Sequential Recommendation