Intention and Context Elicitation with Large Language Models in the Legal Aid Intake Process

Nick Goodson,Rongfei Lu

2023-11-22

Abstract:Large Language Models (LLMs) and chatbots show significant promise in streamlining the legal intake process. This advancement can greatly reduce the workload and costs for legal aid organizations, improving availability while making legal assistance more accessible to a broader audience. However, a key challenge with current LLMs is their tendency to overconfidently deliver an immediate 'best guess' to a client's question based on the output distribution learned over the training data. This approach often overlooks the client's actual intentions or the specifics of their legal situation. As a result, clients may not realize the importance of providing essential additional context or expressing their underlying intentions, which are crucial for their legal cases. Traditionally, logic based decision trees have been used to automate intake for specific access to justice issues, such as immigration and eviction. But those solutions lack scalability. We demonstrate a proof-of-concept using LLMs to elicit and infer clients' underlying intentions and specific legal circumstances through free-form, language-based interactions. We also propose future research directions to use supervised fine-tuning or offline reinforcement learning to automatically incorporate intention and context elicitation in chatbots without explicit prompting.

Computers and Society,Computation and Language

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to effectively obtain the potential intentions and specific legal situations of clients using large - language models (LLMs) during the initial consultation process of legal aid. Current LLMs tend to confidently give a "best guess" as an immediate response to clients' questions based on the output distribution in the training data. This method often ignores the actual intentions of clients or the specific details of their legal situations. As a result, clients may not realize the importance of providing necessary additional background or expressing their potential intentions, which are crucial for their legal cases. Although the traditional logical decision - tree method can automate the handling of specific legal issues (such as immigration and deportation), it lacks scalability. For this reason, this paper proposes a proof - of - concept scheme, using LLMs to extract and infer the potential intentions and specific legal environments of clients through free - form language interactions, and proposes a future research direction, that is, automatically integrating intention and background extraction into chatbots through supervised fine - tuning or offline reinforcement learning without explicit prompts.