Abstract:Modeling user preferences has been mainly addressed by looking at users' interaction history with the different elements available in the system. Tailoring content to individual preferences based on historical data is the main goal of sequential recommendation. The nature of the problem, as well as the good performance observed across various domains, has motivated the use of the transformer architecture, which has proven effective in leveraging increasingly larger amounts of training data when accompanied by an increase in the number of model parameters. This scaling behavior has brought a great deal of attention, as it provides valuable guidance in the design and training of even larger models. Taking inspiration from the scaling laws observed in training large language models, we explore similar principles for sequential recommendation. We use the full Amazon Product Data dataset, which has only been partially explored in other studies, and reveal scaling behaviors similar to those found in language models. Compute-optimal training is possible but requires a careful analysis of the compute-performance trade-offs specific to the application. We also show that performance scaling translates to downstream tasks by fine-tuning larger pre-trained models on smaller task-specific domains. Our approach and findings provide a strategic roadmap for model training and deployment in real high-dimensional preference spaces, facilitating better training and inference efficiency. We hope this paper bridges the gap between the potential of transformers and the intrinsic complexities of high-dimensional sequential recommendation in real-world recommender systems. Code and models can be found at <a class="link-external link-https" href="https://github.com/mercadolibre/srt" rel="external noopener nofollow">this https URL</a>

Scaling Session-Based Transformer Recommendations using Optimized Negative Sampling and Loss Functions

Pareto Front Approximation for Multi-Objective Session-Based Recommender Systems

Scaling Sequential Recommendation Models with Transformers

Recurrent Neural Networks with Top-k Gains for Session-based Recommendations

Personalized Negative Reservoir for Incremental Learning in Recommender Systems

TT-Rec: Tensor Train Compression for Deep Learning Recommendation Models

TRRS: Temporal Recurrent Recommender System based on Time-sync Comments

Improved Recurrent Neural Networks for Session-based Recommendations

Transition Relation Aware Self-Attention for Session-based Recommendation

Optimizing Encoder-Only Transformers for Session-Based Recommendation Systems

STRec: Sparse Transformer for Sequential Recommendations.

Enhancing Transformers without Self-supervised Learning: A Loss Landscape Perspective in Sequential Recommendation

Sentiment-Time Heterogeneous Residual Graph Attention Transformer for Session-Based Recommendation

NineRec: A Benchmark Dataset Suite for Evaluating Transferable Recommendation

Predictive Analysis by Leveraging Temporal User Behavior and User Embeddings

Understanding the Ranking Loss for Recommendation with Sparse User Feedback

Non-Stationary Transformer Architecture: A Versatile Framework for Recommendation Systems

Diversifying Sequential Recommendation with Retrospective and Prospective Transformers

Sequential recommendation: A study on transformers, nearest neighbors and sampled metrics

ContentCTR: Frame-level Live Streaming Click-Through Rate Prediction with Multimodal Transformer

Cracking the Code of Negative Transfer: A Cooperative Game Theoretic Approach for Cross-Domain Sequential Recommendation