Enabling Discriminative Reasoning in LLMs for Legal Judgment Prediction

Chenlong Deng,Kelong Mao,Yuyao Zhang,Zhicheng Dou

2024-08-06

Abstract:Legal judgment prediction is essential for enhancing judicial efficiency. In this work, we identify that existing large language models (LLMs) underperform in this domain due to challenges in understanding case complexities and distinguishing between similar charges. To adapt LLMs for effective legal judgment prediction, we introduce the Ask-Discriminate-Predict (ADAPT) reasoning framework inspired by human judicial reasoning. ADAPT involves decomposing case facts, discriminating among potential charges, and predicting the final judgment. We further enhance LLMs through fine-tuning with multi-task synthetic trajectories to improve legal judgment prediction accuracy and efficiency under our ADAPT framework. Extensive experiments conducted on two widely-used datasets demonstrate the superior performance of our framework in legal judgment prediction, particularly when dealing with complex and confusing charges.

Computation and Language

What problem does this paper attempt to address?

The paper primarily addresses the challenges in the field of Legal Judgment Prediction (LJP), particularly the poor performance of large language models (LLMs) in handling complex case facts and distinguishing similar charges.

Specifically, the paper points out two main challenges that current large language models face in legal judgment prediction:
1. **Understanding case facts**: Extracting key information from complex case descriptions.
2. **Distinguishing similar charges**: Identifying the correct charge labels among multiple potentially similar or overlapping key actions.

To address these issues, the authors propose an inference framework named **Ask-Discrimin Ate-PredicT (ADAPT)**. The design of this framework is inspired by the thought process of human judges when hearing cases, i.e., using legal knowledge to shuttle between case facts and norms. The ADAPT framework includes three steps:
- **Ask**: Decompose the case description into multiple aspects to clarify key criminal facts.
- **Discriminate**: Generate the most likely charge candidates and further evaluate the consistency of each candidate with the case facts.
- **Predict**: Synthesize the reasoning process from the previous two steps to provide the final judgment prediction.

Additionally, to improve the model's performance under the ADAPT framework, the authors propose a method of enhancing LLMs through fine-tuning. This method includes generating high-quality synthetic reasoning trajectories and using these trajectories to fine-tune a smaller model to achieve more accurate and efficient legal judgment predictions.

In summary, this paper aims to enhance the performance of large language models in legal judgment prediction tasks by proposing the ADAPT inference framework and improved fine-tuning methods, especially in handling complex cases and distinguishing similar charges.