Abstract:Lifelong user behavior sequences, comprising up to tens of thousands of history behaviors, are crucial for capturing user interests and predicting user responses in modern recommendation systems. A two-stage paradigm is typically adopted to handle these long sequences: a few relevant behaviors are first searched from the original long sequences via an attention mechanism in the first stage and then aggregated with the target item to construct a discriminative representation for prediction in the second stage. In this work, we identify and characterize, for the first time, a neglected deficiency in existing long-sequence recommendation models: a single set of embeddings struggles with learning both attention and representation, leading to interference between these two processes. Initial attempts to address this issue using linear projections -- a technique borrowed from language processing -- proved ineffective, shedding light on the unique challenges of recommendation models. To overcome this, we propose the Decoupled Attention and Representation Embeddings (DARE) model, where two distinct embedding tables are initialized and learned separately to fully decouple attention and representation. Extensive experiments and analysis demonstrate that DARE provides more accurate search of correlated behaviors and outperforms baselines with AUC gains up to 0.9% on public datasets and notable online system improvements. Furthermore, decoupling embedding spaces allows us to reduce the attention embedding dimension and accelerate the search procedure by 50% without significant performance impact, enabling more efficient, high-performance online serving.

Two-stage Sequential Recommendation Via Bidirectional Attentive Behavior Embedding and Long/Short-term Integration

Sequential Modeling of Hierarchical User Intention and Preference for Next-item Recommendation

Long- and Short- Term Sequential Recommendation with Temporal Interval.

Sequence-Aware Recommendation with Long-Term and Short-Term Attention Memory Networks

Attention with Long-Term Interval-Based Deep Sequential Learning for Recommendation.

Deep Bi-LSTM Networks for Sequential Recommendation

Attentive Hybrid Recurrent Neural Networks for sequential recommendation

Novel Behavior-Enhanced Long- and Short-Term Interest Model for Sequential Recommendation

Dynamic Memory based Attention Network for Sequential Recommendation

Long-Sequence Recommendation Models Need Decoupled Embeddings

A Sequential Recommendation Model for Balancing Long- and Short-Term Benefits

Next-item Recommendation with Bidirectional Encoder Representations from Transformer and Matrix Factorization

Learn over Past, Evolve for Future: Search-based Time-aware Recommendation with Sequential Behavior Data

Learning from History and Present: Next-item Recommendation via Discriminatively Exploiting User Behaviors

Rethinking Lifelong Sequential Recommendation with Incremental Multi-Interest Attention

Time-aware Sequence Model for Next-Item Recommendation.

An Improved Sequential Recommendation Algorithm based on Short-Sequence Enhancement and Temporal Self-Attention Mechanism

Match4Rec: A Novel Recommendation Algorithm Based on Bidirectional Encoder Representation with the Matching Task

Attention And Convolution Enhanced Memory Network For Sequential Recommendation

Adaptive User Modeling with Long and Short-Term Preferences for Personalized Recommendation.