Abstract:Sequential Recommendation (SR) navigates users' dynamic preferences through modeling their historical interactions. The incorporation of the popular Transformer framework, which captures long relationships through pairwise dot products, has notably benefited SR. However, prevailing research in this domain faces three significant challenges: (i) Existing studies directly adopt the primary component of Transformer (i.e., the self-attention mechanism), without a clear explanation or tailored definition for its specific role in SR; (ii) The predominant focus on pairwise computations overlooks the global context or relative prevalence of item pairs within the overall sequence; (iii) Transformer primarily pursues relevance-dominated relationships, neglecting another essential objective in recommendation, i.e., diversity. In response, this work introduces a fresh perspective to elucidate the attention mechanism in SR. Here, attention is defined as dependency interactions among items, quantitatively determined under a global probabilistic model by observing the probabilities of corresponding item subsets. This viewpoint offers a precise and context-specific definition of attention, leading to the design of a distinctive attention mechanism tailored for SR. Specifically, we transmute the well-formulated global, repulsive interactions in Determinantal Point Processes (DPPs) to effectively model dependency interactions. Guided by the repulsive interactions, a theoretically and practically feasible DPP kernel is designed, enabling our attention mechanism to directly consider category/topic distribution for enhancing diversity. Consequently, the Probabilistic Attention mechanism (PAtt) for sequential recommendation is developed. Experimental results demonstrate the excellent scalability and adaptability of our attention mechanism, which significantly improves recommendation performance in terms of both relevance and diversity.

Improving Sequential Query Recommendation with Immediate User Feedback

Sequential recommendation: A study on transformers, nearest neighbors and sampled metrics

Exploring Applications of State Space Models and Advanced Training Techniques in Sequential Recommendations: A Comparative Study on Efficiency and Performance

MaTrRec: Uniting Mamba and Transformer for Sequential Recommendation

Exploration, Exploitation, and Engagement in Multi-Armed Bandits with Abandonment

Sample Enrichment via Temporary Operations on Subsequences for Sequential Recommendation

Mamba4Rec: Towards Efficient Sequential Recommendation with Selective State Space Models

Probabilistic Attention for Sequential Recommendation

Data Augmented Sequential Recommendation based on Counterfactual Thinking

Query-based Interactive Recommendation by Meta-Path and Adapted Attention-GRU

AdaMCT: Adaptive Mixture of CNN-Transformer for Sequential Recommendation

AdaRec: Adaptive Sequential Recommendation for Reinforcing Long-term User Engagement

Recommending High-Utility Search Engine Queries Via A Query-Recommending Model

Knowledge-Enhanced Conversational Recommendation via Transformer-based Sequential Modelling

DRDT: Dynamic Reflection with Divergent Thinking for LLM-based Sequential Recommendation

Leveraging Negative Signals with Self-Attention for Sequential Music Recommendation

Knowledge Prompt-tuning for Sequential Recommendation

Adversarial Oracular Seq2seq Learning for Sequential Recommendation

AutoMLP: Automated MLP for Sequential Recommendations

Deep Exploration for Recommendation Systems

KASR: Knowledge-Aware Sequential Recommendation