Knowledge Graph Tuning: Real-time Large Language Model Personalization based on Human Feedback

Jingwei Sun,Zhixu Du,Yiran Chen

2024-05-30

Abstract:Large language models (LLMs) have demonstrated remarkable proficiency in a range of natural language processing tasks. Once deployed, LLMs encounter users with personalized factual knowledge, and such personalized knowledge is consistently reflected through users' interactions with the LLMs. To enhance user experience, real-time model personalization is essential, allowing LLMs to adapt user-specific knowledge based on user feedback during human-LLM interactions. Existing methods mostly require back-propagation to finetune the model parameters, which incurs high computational and memory costs. In addition, these methods suffer from low interpretability, which will cause unforeseen impacts on model performance during long-term use, where the user's personalized knowledge is accumulated <a class="link-external link-http" href="http://extensively.To" rel="external noopener nofollow">this http URL</a> address these challenges, we propose Knowledge Graph Tuning (KGT), a novel approach that leverages knowledge graphs (KGs) to personalize LLMs. KGT extracts personalized factual knowledge triples from users' queries and feedback and optimizes KGs without modifying the LLM parameters. Our method improves computational and memory efficiency by avoiding back-propagation and ensures interpretability by making the KG adjustments comprehensible to humans.Experiments with state-of-the-art LLMs, including GPT-2, Llama2, and Llama3, show that KGT significantly improves personalization performance while reducing latency and GPU memory costs. Ultimately, KGT offers a promising solution of effective, efficient, and interpretable real-time LLM personalization during user interactions with the LLMs.

Artificial Intelligence

What problem does this paper attempt to address?

### Problems Addressed by the Paper

The paper primarily addresses the personalization issues encountered during the deployment of large-scale language models (LLMs). Specifically:

1. **Personalization Needs**: After deployment, LLMs encounter users with personalized factual knowledge. Users' feedback and queries continuously reflect this personalized knowledge. To enhance user experience, real-time model personalization is required.

2. **Limitations of Existing Methods**: Most existing personalization methods rely on backpropagation to fine-tune model parameters, which incurs high computational and memory costs. Additionally, these methods lack interpretability and may have unforeseen impacts on model performance over long-term use.

### Proposed Method

To address the above challenges, the paper proposes the "Knowledge Graph Tuning" (KGT) method. This method achieves LLM personalization by leveraging knowledge graphs (KG) instead of directly modifying LLM parameters. KGT extracts personalized factual triples from users' queries and feedback and optimizes the KG without altering LLM parameters. This approach improves computational and memory efficiency and ensures the interpretability of the tuning process.

### Experimental Results

The paper conducted experiments on multiple datasets, including advanced pre-trained LLMs such as GPT-2, Llama2, and Llama3. The experimental results show that KGT significantly enhances personalization performance while reducing latency and GPU memory costs by 84% and 77%, respectively. Additionally, KGT demonstrates good scalability in handling a large number of queries, meeting the needs of users accumulating a substantial amount of personalized knowledge.