Abstract:Sequential Recommendation (SR) navigates users' dynamic preferences through modeling their historical interactions. The incorporation of the popular Transformer framework, which captures long relationships through pairwise dot products, has notably benefited SR. However, prevailing research in this domain faces three significant challenges: (i) Existing studies directly adopt the primary component of Transformer (i.e., the self-attention mechanism), without a clear explanation or tailored definition for its specific role in SR; (ii) The predominant focus on pairwise computations overlooks the global context or relative prevalence of item pairs within the overall sequence; (iii) Transformer primarily pursues relevance-dominated relationships, neglecting another essential objective in recommendation, i.e., diversity. In response, this work introduces a fresh perspective to elucidate the attention mechanism in SR. Here, attention is defined as dependency interactions among items, quantitatively determined under a global probabilistic model by observing the probabilities of corresponding item subsets. This viewpoint offers a precise and context-specific definition of attention, leading to the design of a distinctive attention mechanism tailored for SR. Specifically, we transmute the well-formulated global, repulsive interactions in Determinantal Point Processes (DPPs) to effectively model dependency interactions. Guided by the repulsive interactions, a theoretically and practically feasible DPP kernel is designed, enabling our attention mechanism to directly consider category/topic distribution for enhancing diversity. Consequently, the Probabilistic Attention mechanism (PAtt) for sequential recommendation is developed. Experimental results demonstrate the excellent scalability and adaptability of our attention mechanism, which significantly improves recommendation performance in terms of both relevance and diversity.

AdaMCT: Adaptive Mixture of CNN-Transformer for Sequential Recommendation

Attention Calibration for Transformer-based Sequential Recommendation

MaTrRec: Uniting Mamba and Transformer for Sequential Recommendation

Sequential Recommendation with Auxiliary Item Relationships via Multi-Relational Transformer

Improving Transformer-based Sequential Recommenders through Preference Editing

Mamba4Rec: Towards Efficient Sequential Recommendation with Selective State Space Models

STRec: Sparse Transformer for Sequential Recommendations.

An Empirical Study of Training ID-Agnostic Multi-modal Sequential Recommenders

MISSRec: Pre-training and Transferring Multi-modal Interest-aware Sequence Representation for Recommendation

Probabilistic Attention for Sequential Recommendation

Modeling and predicting user preferences with multiple item attributes for sequential recommendations

Multi-Grained Preference Enhanced Transformer for Multi-Behavior Sequential Recommendation

Equivariant Contrastive Learning for Sequential Recommendation

Dual Contrastive Transformer for Hierarchical Preference Modeling in Sequential Recommendation

An Attentive Inductive Bias for Sequential Recommendation beyond the Self-Attention

SSE-PT: Sequential Recommendation Via Personalized Transformer

Sequential Recommendation with Bidirectional Chronological Augmentation of Transformer

Multi-Behavior Hypergraph-Enhanced Transformer for Sequential Recommendation

Pay Attention to Attention for Sequential Recommendation

Online Distillation-enhanced Multi-modal Transformer for Sequential Recommendation

Empowering Sequential Recommendation from Collaborative Signals and Semantic Relatedness