Abstract:Leveraging users' long engagement histories is essential for personalized content recommendations. The success of pretrained language models (PLMs) in NLP has led to their use in encoding user histories and candidate items, framing content recommendations as textual semantic matching tasks. However, existing works still struggle with processing very long user historical text and insufficient user-item interaction. In this paper, we introduce a content-based recommendation framework, SPAR, which effectively tackles the challenges of holistic user interest extraction from the long user engagement history. It achieves so by leveraging PLM, poly-attention layers and attention sparsity mechanisms to encode user's history in a session-based manner. The user and item side features are sufficiently fused for engagement prediction while maintaining standalone representations for both sides, which is efficient for practical model deployment. Moreover, we enhance user profiling by exploiting large language model (LLM) to extract global interests from user engagement history. Extensive experiments on two benchmark datasets demonstrate that our framework outperforms existing state-of-the-art (SoTA) methods.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the challenges faced by existing personalized content recommendation systems when dealing with users' long - term interaction histories, especially how to effectively extract comprehensive user interests from users' long - term interaction histories and enhance the interaction between users and candidate items. Specifically, the paper focuses on the following aspects: 1. **Handling very long user - history texts**: When dealing with users' long - term interaction histories, existing methods result in excessive consumption of memory and computing resources due to the overly long sequence length, making it difficult to effectively extract fine - grained information. The paper proposes a session - based attention sparsity mechanism to alleviate this problem. 2. **Insufficient user - item interaction**: When fusing user and item features, existing methods often rely on separately encoding the histories of users and items and lack cross - content interaction. By introducing the multi - head attention mechanism and post - fusion strategy, the paper enhances the interaction between users and items while maintaining independent representations for practical deployment. 3. **Improving the performance of the recommendation system**: Through the above - mentioned improvements, the paper aims to improve the performance of the recommendation system, especially its performance on large - scale datasets. Experimental results show that the proposed framework outperforms existing state - of - the - art methods on two benchmark datasets. ### Main Contributions 1. **Proposing the SPAR framework**: Combining multiple multi - head attention layers and an attention sparsity mechanism, hierarchically fusing word - level embeddings of session - based user - history texts, effectively extracting user - interest embeddings from long - history - text sequences, and achieving sufficient interaction between users and candidate items. 2. **Experimentally proving the effectiveness**: Tests were carried out on two widely - used datasets (MIND news recommendation and Goodreads book recommendation), and the results show that SPAR improves the AUC scores by 1.48% and 1.15% respectively, significantly outperforming existing methods. 3. **Extensive ablation studies**: Through detailed ablation studies, the influence of each component in the framework is demonstrated, providing insights into potential trade - offs when designing content recommendation systems. ### Method Overview - **User - history encoder**: Use a pre - trained language model (PLM) to encode users' long - term interaction histories and reduce computational complexity through session segmentation and an attention sparsity mechanism. - **User - interest extraction**: Utilize a large - language model (LLM) to generate user - interest summaries, further enriching the user - history representation. - **Candidate - content encoder**: Share the PLM to encode candidate content and generate multiple candidate - content representations through the multi - head attention mechanism. - **Interaction predictor**: Calculate the matching score between user and candidate - content embeddings through a lightweight attention layer for click - through - rate (CTR) prediction. In conclusion, through a series of innovative technical means, this paper solves the key problems in personalized content recommendation systems and significantly improves the performance of the recommendation system.

SPAR: Personalized Content-Based Recommendation via Long Engagement Attention

Self-Attention Mechanism Enhanced User Interests Modeling for Personalized Recommendation Services in Cyber-Physical-Social Systems.

Spatial and Temporal User Interest Representations for Sequential Recommendation

Exploiting Structural and Temporal Influence for Dynamic Social-Aware Recommendation

Disentangling Long and Short-Term Interests for Recommendation

Long and Short-Term User Intent Learning for Sequence Recommendation

Long- and short-term collaborative attention networks for sequential recommendation

Adaptive User Modeling with Long and Short-Term Preferences for Personalized Recommendation.

Modeling Long-Term and Short-Term Interests with Parallel Attentions for Session-Based Recommendation

PRECISE: Pre-training Sequential Recommenders with Collaborative and Semantic Information

PALR: Personalization Aware LLMs for Recommendation

Group-based Personalized News Recommendation with Long- and Short-term Fine-Grained Matching

LLM-ESR: Large Language Models Enhancement for Long-tailed Sequential Recommendation

Long-short Interest Network with Graph-Based Method for Sequential Recommendation

EmbSum: Leveraging the Summarization Capabilities of Large Language Models for Content-Based Recommendations

Harnessing Large Language Models for Text-Rich Sequential Recommendation

SEMINAR: Search Enhanced Multi-modal Interest Network and Approximate Retrieval for Lifelong Sequential Recommendation

Personalized Recommendation Systems Powered By Large Language Models: Integrating Semantic Understanding and User Preferences

Two-stage Sequential Recommendation Via Bidirectional Attentive Behavior Embedding and Long/Short-term Integration

LLMs for User Interest Exploration in Large-scale Recommendation Systems

Recommendation Based on Users’ Long-Term and Short-Term Interests with Attention