Knowledge-Infused Legal Wisdom: Navigating LLM Consultation through the Lens of Diagnostics and Positive-Unlabeled Reinforcement Learning

Yang Wu,Chenghao Wang,Ece Gumusel,Xiaozhong Liu

2024-06-06

Abstract:The integration of generative Large Language Models (LLMs) into various applications, including the legal domain, has been accelerated by their expansive and versatile nature. However, when facing a legal case, users without a legal background often struggle to formulate professional queries and may inadvertently overlook critical legal factors when presenting their case narrative to LLMs. To address this issue, we propose the Diagnostic Legal Large Language Model (D3LM), which utilizes adaptive lawyer-like diagnostic questions to collect additional case information and then provides high-quality feedback. D3LM incorporates an innovative graph-based Positive-Unlabeled Reinforcement Learning (PURL) algorithm, enabling the generation of critical questions and enhancing user-LLM interactions. Moreover, an integrated LLM-based stopping criterion facilitates precise Court Views Generation (CVG). Our research also introduces a new English-language CVG dataset based on the US case law database, enriching the realm of LLM research and deployment with a vital dimension. D3LM surpasses classical LLMs by delivering outstanding performance and a remarkable user experience in the legal domain.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

This paper aims to address the problem that users have difficulty raising professional legal questions when interacting with large-scale language models (LLMs) without a legal background. When users describe legal cases, they may overlook key details, which can affect the LLMs' ability to provide effective assistance. To solve this problem, the paper proposes a "Diagnostic Legal Large Language Model" (D3LM) that uses lawyer-like adaptive diagnostic questions to gather more information and provide legal advice through high-quality feedback. D3LM adopts a graph-based Positive Unlabeled Reinforcement Learning (PURL) algorithm to generate important questions and enhance user interaction with LLMs. In addition, the research introduces a new English Commonly Voiced as Graphs (CVG) dataset based on the American case law database, enriching the LLM research field.

D3LM surpasses traditional LLMs in the following ways:
1. Active questioning: Imitates lawyer consultation strategies to obtain detailed case information through targeted questions.
2. PURL algorithm: Dynamically identifies key factors and adaptsively generates questions to improve information gathering ability.
3. New dataset: A new dataset focusing on American legal cases fills the gap in English legal resources.

Through these innovations, D3LM improves the prediction accuracy of the legal field and provides more efficient and cost-effective legal insights, potentially reforming the way legal services are provided.