Adversarially Robust Neural Legal Judgement Systems

Rohit Raj,V Susheela Devi

2023-08-01

Abstract:Legal judgment prediction is the task of predicting the outcome of court cases on a given text description of facts of cases. These tasks apply Natural Language Processing (NLP) techniques to predict legal judgment results based on facts. Recently, large-scale public datasets and NLP models have increased research in areas related to legal judgment prediction systems. For such systems to be practically helpful, they should be robust from adversarial attacks. Previous works mainly focus on making a neural legal judgement system; however, significantly less or no attention has been given to creating a robust Legal Judgement Prediction(LJP) system. We implemented adversarial attacks on early existing LJP systems and found that none of them could handle attacks. In this work, we proposed an approach for making robust LJP systems. Extensive experiments on three legal datasets show significant improvements in our approach over the state-of-the-art LJP system in handling adversarial attacks. To the best of our knowledge, we are the first to increase the robustness of early-existing LJP systems.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

The paper primarily focuses on the robustness of legal judgment prediction systems, particularly their performance under adversarial attacks. Specifically, the researchers found that existing legal judgment prediction systems (Legal Judgment Prediction, LJP), although capable of predicting court case outcomes with the help of natural language processing (NLP) techniques, perform poorly when faced with deliberately constructed adversarial examples. These adversarial examples are generated by subtly modifying the input text, aiming to mislead the model while being almost imperceptible to human readers.

The main contributions of the paper include:
1. **Adversarial Attack Experiments**: The researchers first implemented adversarial attacks to test the performance of existing baseline models (such as BERT, Legal-BERT, etc.) in adversarial environments and found that these models are susceptible to attacks, with their performance significantly declining.
2. **Proposed Algorithm**: Based on the above findings, the paper proposes an algorithm for training adversarially robust legal models, aiming to improve the models' performance when facing adversarial examples.
3. **Enhanced Training Methods**: The researchers also employed data augmentation and adversarial training methods to further enhance the robustness of the models.

The experimental section demonstrates the effectiveness of the proposed method through extensive experiments on four different legal datasets. The results show that the proposed approach significantly improves the handling of adversarial attacks compared to existing techniques. Additionally, the paper provides detailed experimental settings and result analyses to prove the effectiveness and importance of their method. Overall, this work aims to address the shortcomings of existing legal judgment prediction systems in terms of robustness and provide solutions for security challenges that may be encountered in actual deployment.