Abstract:Sequential Recommendation (SR) navigates users' dynamic preferences through modeling their historical interactions. The incorporation of the popular Transformer framework, which captures long relationships through pairwise dot products, has notably benefited SR. However, prevailing research in this domain faces three significant challenges: (i) Existing studies directly adopt the primary component of Transformer (i.e., the self-attention mechanism), without a clear explanation or tailored definition for its specific role in SR; (ii) The predominant focus on pairwise computations overlooks the global context or relative prevalence of item pairs within the overall sequence; (iii) Transformer primarily pursues relevance-dominated relationships, neglecting another essential objective in recommendation, i.e., diversity. In response, this work introduces a fresh perspective to elucidate the attention mechanism in SR. Here, attention is defined as dependency interactions among items, quantitatively determined under a global probabilistic model by observing the probabilities of corresponding item subsets. This viewpoint offers a precise and context-specific definition of attention, leading to the design of a distinctive attention mechanism tailored for SR. Specifically, we transmute the well-formulated global, repulsive interactions in Determinantal Point Processes (DPPs) to effectively model dependency interactions. Guided by the repulsive interactions, a theoretically and practically feasible DPP kernel is designed, enabling our attention mechanism to directly consider category/topic distribution for enhancing diversity. Consequently, the Probabilistic Attention mechanism (PAtt) for sequential recommendation is developed. Experimental results demonstrate the excellent scalability and adaptability of our attention mechanism, which significantly improves recommendation performance in terms of both relevance and diversity.

Learning Positional Attention for Sequential Recommendation

Position-aware Context Attention for Session-Based Recommendation.

Learning Self-Modulating Attention In Continuous Time Space With Applications To Sequential Recommendation

Attention with Long-Term Interval-Based Deep Sequential Learning for Recommendation.

Probabilistic Attention for Sequential Recommendation

Self-Attentive Sequential Recommendation

Time Interval Aware Self-Attention for Sequential Recommendation

Adaptive Attention-Aware Gated Recurrent Unit for Sequential Recommendation

EASRec: External Attentive Efficient Sequential Recommender

A Time-Aware Self-Attention Based Neural Network Model for Sequential Recommendation.

MLP4Rec: A Pure MLP Architecture for Sequential Recommendations

Positional encoding is not the same as context: A study on positional encoding for Sequential recommendation

Intention-Centric Learning via Dual Attention for Sequential Recommendation

Long-short Interest Network with Graph-Based Method for Sequential Recommendation

Context-Aware POI Sequence Recommendation with Attention-Based Neural Network

Recursive Attentive Methods with Reused Item Representations for Sequential Recommendation

Long-Sequence Recommendation Models Need Decoupled Embeddings

Attentive Hawkes Process Application for Sequential Recommendation

Pay Attention to Attention for Sequential Recommendation

FSASA: Sequential recommendation based on fusing session-aware models and self-attention networks

HSA: Hyperbolic Self-attention for Sequential Recommendation