Abstract:Sequential recommendation systems predict the next interaction item based on users' past interactions, aligning recommendations with individual preferences. Leveraging the strengths of Large Language Models (LLMs) in knowledge comprehension and reasoning, recent approaches are eager to apply LLMs to sequential recommendation. A common paradigm is converting user behavior sequences into instruction data, and fine-tuning the LLM with parameter-efficient fine-tuning (PEFT) methods like Low-Rank Adaption (LoRA). However, the uniform application of LoRA across diverse user behaviors is insufficient to capture individual variability, resulting in negative transfer between disparate sequences. To address these challenges, we propose Instance-wise LoRA (iLoRA). We innovatively treat the sequential recommendation task as a form of multi-task learning, integrating LoRA with the Mixture of Experts (MoE) framework. This approach encourages different experts to capture various aspects of user behavior. Additionally, we introduce a sequence representation guided gate function that generates customized expert participation weights for each user sequence, which allows dynamic parameter adjustment for instance-wise recommendations. In sequential recommendation, iLoRA achieves an average relative improvement of 11.4\% over basic LoRA in the hit ratio metric, with less than a 1\% relative increase in trainable parameters. Extensive experiments on three benchmark datasets demonstrate the effectiveness of iLoRA, highlighting its superior performance compared to existing methods in mitigating negative transfer and improving recommendation accuracy. Our data and code are available at <a class="link-external link-https" href="https://github.com/AkaliKong/iLoRA" rel="external noopener nofollow">this https URL</a>.

RAPO: an Adaptive Ranking Paradigm for Bilingual Lexicon Induction

RAPO: An Adaptive Ranking Paradigm for Bilingual Lexicon Induction

How Lexical is Bilingual Lexicon Induction?

L2R²: Leveraging Ranking for Abductive Reasoning

Improving Bilingual Lexicon Induction with Cross-Encoder Reranking

Bilingual Lexicon Induction from Non-Parallel Data with Minimal Supervision.

Low-Rank Adaptation for Multilingual Summarization: An Empirical Study

Choosing Transfer Languages for Cross-Lingual Learning

RA-Rec: An Efficient ID Representation Alignment Framework for LLM-based Recommendation

Unsupervised Bilingual Lexicon Induction Via Latent Variable Models.

DAMO-NLP at SemEval-2023 Task 2: A Unified Retrieval-augmented System for Multilingual Named Entity Recognition

Customizing Language Models with Instance-wise LoRA for Sequential Recommendation

Simple Yet Effective Neural Ranking and Reranking Baselines for Cross-Lingual Information Retrieval

ALoRA: Allocating Low-Rank Adaptation for Fine-tuning Large Language Models

R 2 : A Novel Recall & Ranking Framework for Legal Judgment Prediction

DemoRank: Selecting Effective Demonstrations for Large Language Models in Ranking Task

Ranking With Recursive Neural Networks And Its Application To Multi-Document Summarization

Reindex-Then-Adapt: Improving Large Language Models for Conversational Recommendation

Cross-language Document Summarization Via Extraction and Ranking of Multiple Summaries

RRescue: Ranking LLM Responses to Enhance Reasoning over Context

PaRaDe: Passage Ranking using Demonstrations with Large Language Models