Abstract:Sequential recommendation represents a pivotal branch of recommendation systems, centered around dynamically analyzing the sequential dependencies between user preferences and their interactive behaviors. Despite the Transformer architecture-based models achieving commendable performance within this domain, their quadratic computational complexity relative to the sequence dimension impedes efficient modeling. In response, the innovative Mamba architecture, characterized by linear computational complexity, has emerged. Mamba4Rec further pioneers the application of Mamba in sequential recommendation. Nonetheless, Mamba 1's hardware-aware algorithm struggles to efficiently leverage modern matrix computational units, which lead to the proposal of the improved State Space Duality (SSD), also known as Mamba 2. While the SSD4Rec successfully adapts the SSD architecture for sequential recommendation, showing promising results in high-dimensional contexts, it suffers significant performance drops in low-dimensional scenarios crucial for pure ID sequential recommendation tasks. Addressing this challenge, we propose a novel sequential recommendation backbone model, TiM4Rec, which ameliorates the low-dimensional performance loss of the SSD architecture while preserving its computational efficiency. Drawing inspiration from TiSASRec, we develop a time-aware enhancement method tailored for the linear computation demands of the SSD architecture, thereby enhancing its adaptability and achieving state-of-the-art (SOTA) performance in both low and high-dimensional modeling. The code for our model is publicly accessible at <a class="link-external link-https" href="https://github.com/AlwaysFHao/TiM4Rec" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The problem this paper attempts to address is the poor performance of existing State Space Duality (SSD) models in low-dimensional feature spaces, especially in ID-based sequential recommendation tasks. While the SSD architecture performs well in high-dimensional feature spaces, its performance significantly declines in low-dimensional scenarios, limiting its broad applicability in practical applications. Specifically, the paper points out that although the SSD architecture has the advantage of linear computational complexity, its performance in low-dimensional feature spaces is inferior to that of traditional State Space Model (SSM) architectures. To overcome this shortcoming, the authors propose a new time-aware enhancement method, the TiM4Rec model, which improves the performance of the SSD architecture in low-dimensional feature spaces by introducing a time-aware mask matrix while maintaining its computational efficiency. The main contributions of the paper include: 1. **Time-aware enhancement method**: For the first time, a time-aware enhancement method suitable for the Mamba architecture is explored, specifically targeting sequential recommendation tasks with linear computational complexity. 2. **TiM4Rec model**: By introducing the proposed time-aware enhancement method, the TiM4Rec model is developed, effectively reducing the performance loss of the SSD architecture in low-dimensional spaces and improving its adaptability and training efficiency in ID modeling tasks. 3. **Experimental validation**: Comprehensive experiments on 3 datasets demonstrate that the TiM4Rec model performs excellently in low-dimensional spaces, outperforming the SSD4Rec model, while maintaining the advantages of the SSD architecture in high-dimensional scenarios. In summary, this paper aims to improve the SSD architecture through a time-aware enhancement method, enabling it to efficiently perform sequential recommendation tasks in both low-dimensional and high-dimensional feature spaces.

TiM4Rec: An Efficient Sequential Recommendation Model Based on Time-Aware Structured State Space Duality Model

SSD4Rec: A Structured State Space Duality Model for Efficient Sequential Recommendation

Mamba4Rec: Towards Efficient Sequential Recommendation with Selective State Space Models

MaTrRec: Uniting Mamba and Transformer for Sequential Recommendation

EchoMamba4Rec: Harmonizing Bidirectional State Space Models with Spectral Filtering for Advanced Sequential Recommendation

SIGMA: Selective Gated Mamba for Sequential Recommendation

A Time-Aware Self-Attention Based Neural Network Model for Sequential Recommendation.

Bidirectional Gated Mamba for Sequential Recommendation

Uncovering Selective State Space Model's Capabilities in Lifelong Sequential Recommendation

SMLP4Rec: An Efficient all-MLP Architecture for Sequential Recommendations

Exploring Applications of State Space Models and Advanced Training Techniques in Sequential Recommendations: A Comparative Study on Efficiency and Performance

MLSA4Rec: Mamba Combined with Low-Rank Decomposed Self-Attention for Sequential Recommendation

TTT4Rec: A Test-Time Training Approach for Rapid Adaption in Sequential Recommendation

GAT4Rec: Sequential Recommendation with a Gated Recurrent Unit and Transformers

DTR4Rec: Direct Transition Relationship for Sequential Recommendation

Recognizing Semantics-Consistent Subsequences for Sequential Recommendation

Continuous-Time Sequential Recommendation with Temporal Graph Collaborative Transformer

MLP4Rec: A Pure MLP Architecture for Sequential Recommendations

S3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization

CT4Rec: Simple Yet Effective Consistency Training for Sequential Recommendation.

AdaMCT: Adaptive Mixture of CNN-Transformer for Sequential Recommendation