Abstract:We primarily focus on the field of large language models (LLMs) for recommendation, which has been actively explored recently and poses a significant challenge in effectively enhancing recommender systems with logical reasoning abilities and open-world knowledge. Current mainstream efforts mainly center around injecting personalized information from recommendation models into LLMs by customizing input templates or aligning representations between semantic and recommendation spaces at the prediction layer. However, they face three significant limitations: (1) LoRA is mostly used as a core component in existing works, but personalization is not well established in LoRA parameters as the LoRA matrix shared by every user may not cater to different users' characteristics, leading to suboptimal performance. (2) Although lifelong personalized behavior sequences are ideal for personalization, their use raises effectiveness and efficiency issues since LLMs require escalating training and inference time to extend text lengths. (3) Existing approaches aren't scalable for large datasets due to training efficiency constraints. Thus, LLMs only see a small fraction of the datasets (e.g., less than 10%) instead of the whole datasets, limiting their exposure to the full training space. To address these problems, we propose RecLoRA. This model incorporates a Personalized LoRA module that maintains independent LoRAs for different users and a Long-Short Modality Retriever that retrieves different history lengths for different modalities, significantly improving performance while adding minimal time cost. Furthermore, we design a Few2Many Learning Strategy, using a conventional recommendation model as a lens to magnify small training spaces to full spaces. Extensive experiments on public datasets demonstrate the efficacy of our RecLoRA compared to existing baseline models.

MultiLoRA: Multi-Directional Low Rank Adaptation for Multi-Domain Recommendation

MultiLoRA: Democratizing LoRA for Better Multi-Task Learning

Lifelong Personalized Low-Rank Adaptation of Large Language Models for Recommendation

MLoRA: Multi-Domain Low-Rank Adaptive Network for CTR Prediction

MAMDR: A Model Agnostic Learning Method for Multi-Domain Recommendation

Customizing Language Models with Instance-wise LoRA for Sequential Recommendation

HyperLoRA: Efficient Cross-task Generalization Via Constrained Low-Rank Adapters Generation

MTL-LoRA: Low-Rank Adaptation for Multi-Task Learning

MCRPL: A Pretrain, Prompt, and Fine-tune Paradigm for Non-overlapping Many-to-one Cross-domain Recommendation

One for Dozens: Adaptive REcommendation for All Domains with Counterfactual Augmentation

MCRPL: A Pretrain, Prompt & Fine-tune Paradigm for Non-overlapping Many-to-one Cross-domain Recommendation

Multi-Domain Recommendation to Attract Users via Domain Preference Modeling

MoR: Mixture of Ranks for Low-Rank Adaptation Tuning

Finetuning Large Language Model for Personalized Ranking

SuperLoRA: Parameter-Efficient Unified Adaptation of Multi-Layer Attention Modules

Aligning Large Language Model with Direct Multi-Preference Optimization for Recommendation

Mixture-of-LoRAs: An Efficient Multitask Tuning for Large Language Models

One Model for All: Large Language Models Are Domain-Agnostic Recommendation Systems

Adaptive Domain Interest Network for Multi-domain Recommendation

Chain-of-LoRA: Enhancing the Instruction Fine-Tuning Performance of Low-Rank Adaptation on Diverse Instruction Set

AutoRank: MCDA Based Rank Personalization for LoRA-Enabled Distributed Learning