Abstract:Sequential Recommendation (SR) navigates users' dynamic preferences through modeling their historical interactions. The incorporation of the popular Transformer framework, which captures long relationships through pairwise dot products, has notably benefited SR. However, prevailing research in this domain faces three significant challenges: (i) Existing studies directly adopt the primary component of Transformer (i.e., the self-attention mechanism), without a clear explanation or tailored definition for its specific role in SR; (ii) The predominant focus on pairwise computations overlooks the global context or relative prevalence of item pairs within the overall sequence; (iii) Transformer primarily pursues relevance-dominated relationships, neglecting another essential objective in recommendation, i.e., diversity. In response, this work introduces a fresh perspective to elucidate the attention mechanism in SR. Here, attention is defined as dependency interactions among items, quantitatively determined under a global probabilistic model by observing the probabilities of corresponding item subsets. This viewpoint offers a precise and context-specific definition of attention, leading to the design of a distinctive attention mechanism tailored for SR. Specifically, we transmute the well-formulated global, repulsive interactions in Determinantal Point Processes (DPPs) to effectively model dependency interactions. Guided by the repulsive interactions, a theoretically and practically feasible DPP kernel is designed, enabling our attention mechanism to directly consider category/topic distribution for enhancing diversity. Consequently, the Probabilistic Attention mechanism (PAtt) for sequential recommendation is developed. Experimental results demonstrate the excellent scalability and adaptability of our attention mechanism, which significantly improves recommendation performance in terms of both relevance and diversity.

Sequence and Distance Aware Transformer for Recommendation Systems

Towards Efficient and Effective Transformers for Sequential Recommendation.

Adaptive Disentangled Transformer for Sequential Recommendation

STRec: Sparse Transformer for Sequential Recommendations.

A Global Contextual Enhanced Structural-Aware Transformer for Sequential Recommendation

Designing a Sequential Recommendation System for Heterogeneous Interactions Using Transformers

Attention Calibration for Transformer-based Sequential Recommendation

Pay Attention to Attention for Sequential Recommendation

GAT4Rec: Sequential Recommendation with a Gated Recurrent Unit and Transformers

Graph-Coupled Time Interval Network for Sequential Recommendation

SSE-PT: Sequential Recommendation Via Personalized Transformer

MGT: Multi-Granularity Transformer Leveraging Multi-Level Relation for Sequential Recommendation

Enhancing Transformers without Self-supervised Learning: A Loss Landscape Perspective in Sequential Recommendation

Sequence recommendation algorithm fusing filter and transformer under joint training

Sequential Recommendation with Auxiliary Item Relationships via Multi-Relational Transformer

Probabilistic Attention for Sequential Recommendation

Non-Stationary Transformer Architecture: A Versatile Framework for Recommendation Systems

Sequential Recommendation with Bidirectional Chronological Augmentation of Transformer

Continuous-Time Sequential Recommendation with Temporal Graph Collaborative Transformer

AdaMCT: Adaptive Mixture of CNN-Transformer for Sequential Recommendation

Integrating Offline Reinforcement Learning with Transformers for Sequential Recommendation