Junkai Liu,Yujie Tong,Hui Huang,Shuyuan Zheng,Muyun Yang,Peicheng Wu,Makoto Onizuka,Chuan Xiao
Abstract:Legal facts refer to the facts that can be proven by acknowledged evidence in a trial. They form the basis for the determination of court judgments. This paper introduces a novel NLP task: legal fact prediction, which aims to predict the legal fact based on a list of evidence. The predicted facts can instruct the parties and their lawyers involved in a trial to strengthen their submissions and optimize their strategies during the trial. Moreover, since real legal facts are difficult to obtain before the final judgment, the predicted facts also serve as an important basis for legal judgment prediction. We construct a benchmark dataset consisting of evidence lists and ground-truth legal facts for real civil loan cases, LFPLoan. Our experiments on this dataset show that this task is non-trivial and requires further considerable research efforts.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to automatically predict legal facts (legal fact prediction) during the legal litigation process. Specifically, the paper proposes a new natural language processing (NLP) task, that is, to predict the legal facts to be determined by the court based on the list of evidence provided by the parties. These predicted legal facts can guide all parties in the case and their lawyers to strengthen evidence submission and optimize trial strategies, and can also serve as an important basis for legal judgment prediction.
### Background and Significance
In legal litigation, legal facts refer to the facts confirmed by the judge through the evidence submitted by the parties and cross - examination in court. These facts are the basis for the court to make a judgment. However, before the final judgment, the real legal facts are difficult to obtain, so predicting legal facts is of great significance for assisting judicial decision - making.
### Research Status
Previous studies have mainly focused on automated judicial judgment prediction, usually directly using the legal facts recorded in official judgments to predict the final judgment. This method bypasses the crucial legal fact - finding process. In practice, the parties only know the evidence information before the trial, and the legal facts need to be determined through strict cross - examination during the trial process. Therefore, legal fact prediction is a task that urgently needs to be explored and is in high demand.
### Paper Contributions
1. **Task Definition**: The paper formally defines the legal fact prediction task, that is, given the list of evidence provided by the plaintiff and the defendant, predict the legal facts to be determined by the court.
2. **Dataset Construction**: The paper constructs a benchmark dataset LFP - Loan, which contains the list of evidence and real legal facts extracted from the publicly available civil lending case trial records.
3. **Experimental Verification**: The paper tests two baseline methods based on large language models (LLM). The experimental results show that these methods perform well in predicting some relatively clear legal facts, but perform poorly in dealing with the争议点 (dispute points) in cases, which highlights the complexity and challenges of the task.
### Experimental Methods
1. **Baseline Methods**:
- **Question - Answering (QA) Method**: Provide the list of evidence to an LLM agent playing the role of a judge and ask it to determine the legal facts based on the evidence.
- **Simulation Method**: Use multiple LLM agents to simulate the court trial process, with each agent playing the role of a judge, plaintiff or defendant respectively. During the simulated trial, the plaintiff and the defendant clarify and debate the case facts based on the evidence they respectively hold, and finally the judge agent summarizes the legal facts.
2. **Evaluation Metrics**:
- **Fact Description Similarity**: Use OpenAI's text - embedding - 3 - large model to convert the predicted fact description and the real fact description into embedding vectors and calculate the cosine similarity.
- **Key Item Accuracy**: Conduct a strict accuracy assessment on key items such as loan amount, whether to calculate interest, interest amount, loan date, repayment date and amount already repaid.
### Results and Discussion
The experimental results show that although the LLM - based methods show a certain ability in predicting basic facts, the prediction effect on the dispute points in cases (such as loan date, repayment date, amount already repaid, etc.) is poor. This indicates that the legal fact prediction task requires stronger reasoning and the ability to process complex information.
### Conclusion
The paper introduces a new task of legal fact prediction and constructs a benchmark dataset LFP - Loan. Although the preliminary experimental results show that the task is challenging, it still provides directions for future research. By further improving models and methods, it is expected to improve the accuracy and practicality of legal fact prediction.