Leveraging Large Language Models for Learning Complex Legal Concepts through Storytelling

Hang Jiang,Xiajie Zhang,Robert Mahari,Daniel Kessler,Eric Ma,Tal August,Irene Li,Alex 'Sandy' Pentland,Yoon Kim,Deb Roy,Jad Kabbara

2024-07-03

Abstract:Making legal knowledge accessible to non-experts is crucial for enhancing general legal literacy and encouraging civic participation in democracy. However, legal documents are often challenging to understand for people without legal backgrounds. In this paper, we present a novel application of large language models (LLMs) in legal education to help non-experts learn intricate legal concepts through storytelling, an effective pedagogical tool in conveying complex and abstract concepts. We also introduce a new dataset LegalStories, which consists of 294 complex legal doctrines, each accompanied by a story and a set of multiple-choice questions generated by LLMs. To construct the dataset, we experiment with various LLMs to generate legal stories explaining these concepts. Furthermore, we use an expert-in-the-loop approach to iteratively design multiple-choice questions. Then, we evaluate the effectiveness of storytelling with LLMs through randomized controlled trials (RCTs) with legal novices on 10 samples from the dataset. We find that LLM-generated stories enhance comprehension of legal concepts and interest in law among non-native speakers compared to only definitions. Moreover, stories consistently help participants relate legal concepts to their lives. Finally, we find that learning with stories shows a higher retention rate for non-native speakers in the follow-up assessment. Our work has strong implications for using LLMs in promoting teaching and learning in the legal field and beyond.

Computation and Language,Human-Computer Interaction

What problem does this paper attempt to address?

The paper attempts to address the issue of improving the accessibility of legal knowledge for non-professionals. Specifically, the authors focus on how to use large language models (LLMs) to help individuals without a legal background understand and learn complex legal concepts through storytelling. The paper points out that existing legal documents are often difficult for people without a legal background to understand, which hinders the improvement of legal literacy and democratic participation among the general public. Therefore, the authors propose a new method of explaining complex legal concepts through stories generated by LLMs and design a series of multiple-choice questions to assess learning outcomes. Additionally, they have constructed a new dataset called LEGAL STORIES, which contains 294 complex legal principles, each accompanied by a story generated by LLMs and a series of multiple-choice questions. Through this method, the authors aim to verify whether stories generated by LLMs can enhance non-professionals' understanding and interest in legal concepts, especially among non-native speakers, and whether these stories help participants relate legal concepts to their own lives and improve their memory retention in subsequent assessments. This work is of significant importance for promoting teaching and learning in the legal field and improving overall legal literacy.

Leveraging Large Language Models for Learning Complex Legal Concepts through Storytelling

Can Large Language Models Grasp Legal Theories? Enhance Legal Reasoning with Insights from Multi-Agent Collaboration

Optimizing Numerical Estimation and Operational Efficiency in the Legal Domain through Large Language Models

LawLLM: Law Large Language Model for the US Legal System

Large language models as tax attorneys: a case study in legal capabilities emergence

Knowledge-Infused Legal Wisdom: Navigating LLM Consultation through the Lens of Diagnostics and Positive-Unlabeled Reinforcement Learning

Large Language Models are legal but they are not: Making the case for a powerful LegalLLM

Large Language Models and Explainable Law: a Hybrid Methodology

InternLM-Law: An Open Source Chinese Legal Large Language Model

TransformLLM: Adapting Large Language Models via LLM-Transformed Reading Comprehension Text

Using Large Language Models to Support Thematic Analysis in Empirical Legal Studies

Caveat Lector: Large Language Models in Legal Practice

Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models

From Text to Structure: Using Large Language Models to Support the Development of Legal Expert Systems

Exploring New Frontiers of Deep Learning in Legal Practice: A Case Study of Large Language Models

Intention and Context Elicitation with Large Language Models in the Legal Aid Intake Process

Explingo: Explaining AI Predictions using Large Language Models

"It Explains What I am Currently Going Through Perfectly to a Tee": Understanding User Perceptions on LLM-Enhanced Narrative Interventions

Student Perspectives on Using a Large Language Model (LLM) for an Assignment on Professional Ethics

Enhancing Logical Reasoning in Large Language Models to Facilitate Legal Applications

Leveraging Large Language Models for Relevance Judgments in Legal Case Retrieval