Abstract:The current fascination with large language models, or LLMs, derives from the fact that many users lack the expertise to evaluate the quality of the generated text. LLMs may therefore appear more capable than they actually are. The dangerous combination of fluency and superficial plausibility leads to the temptation to trust the generated text and creates the risk of overreliance. Who would not trust perfect legalese? Relying recent findings in both technical and legal scholarship, this Article counterbalances the overly optimistic predictions as to the role of LLMs in legal practice. Integrating LLMs into legal workstreams without a better comprehension of their limitations, will create inefficiencies if not outright risks. Notwithstanding their unprecedented ability to generate text, LLMs do not understand text. Without the ability to understand meaning, LLMs will remain unable to use language, to acquire knowledge and to perform complex reasoning tasks. Trained to model language on the basis of stochastic word predictions, LLMs cannot distinguish fact from fiction. Their knowledge of the law is limited to word strings memorized in their parameters. It is also incomplete and largely incorrect. LLMs operate at the level of word distributions, not at the level of verified facts. The resulting propensity to hallucinate, to produce statements that are incorrect but appear helpful and relevant, is alarming in high-risk areas like legal services. At present, lawyers should beware of relying on text generated by LLMs.

What problem does this paper attempt to address?

The paper attempts to address the risks and limitations that large language models (LLMs) may bring to legal practice. Specifically, the paper focuses on the following aspects: 1. **Misunderstanding of LLMs' capabilities**: Many users lack the expertise to evaluate the quality of text generated by LLMs, which may lead to overestimating these models' capabilities. This overestimation can result in an over-reliance on the text generated by LLMs, ignoring their potential risks. 2. **Limitations of LLMs**: - **Understanding ability**: Although LLMs can generate fluent and seemingly reasonable text, they do not actually understand the meaning of the text. This limits their application in tasks that require deep understanding and reasoning. - **Knowledge acquisition**: The knowledge of LLMs is limited to the phrase memory in the training data, which is often incomplete and inaccurate. They cannot distinguish between fact and fiction. - **Hallucination phenomenon**: LLMs tend to generate incorrect but seemingly reasonable information. This "hallucination" is particularly concerning in high-risk areas such as legal services. 3. **Technical background and limitations**: - The paper details the technical background of LLMs, including how they generate text by predicting the next word. - It emphasizes the limitations of LLMs in handling complex tasks, especially those that require understanding, reasoning, and domain-specific knowledge. 4. **Application in legal practice**: - The paper explores the potential risks of integrating LLMs into legal workflows, pointing out that a lack of full awareness of these models' limitations may lead to inefficiency or even direct risks. - It suggests that lawyers should use the text generated by LLMs cautiously and rigorously verify its content. 5. **Future solutions**: - The paper discusses some possible solutions, including expanding the model size, improving the quality and quantity of training data, and enhancing human intervention and input. - It emphasizes that many of the limitations of LLMs remain open research questions, and there are currently no effective solutions. Overall, the paper aims to remind legal professionals to remain vigilant about the capabilities of LLMs, avoid blindly trusting the text generated by these models, and emphasize caution in practical applications.

Caveat Lector: Large Language Models in Legal Practice

Large Language Models are legal but they are not: Making the case for a powerful LegalLLM

Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models

Objection Overruled! Lay People can Distinguish Large Language Models from Lawyers, but still Favour Advice from an LLM

TransformLLM: Adapting Large Language Models via LLM-Transformed Reading Comprehension Text

Misinforming LLMs: vulnerabilities, challenges and opportunities

Large language models as tax attorneys: a case study in legal capabilities emergence

LLMs' Understanding of Natural Language Revealed

A Comprehensive Evaluation of Large Language Models on Legal Judgment Prediction

The Factuality of Large Language Models in the Legal Domain

Stochastic LLMs do not Understand Language: Towards Symbolic, Explainable and Ontologically Based LLMs

Can LLMs be Fooled? Investigating Vulnerabilities in LLMs

BLT: Can Large Language Models Handle Basic Legal Text?

It Cannot Be Right If It Was Written by AI: On Lawyers' Preferences of Documents Perceived as Authored by an LLM vs a Human

Better Call GPT, Comparing Large Language Models Against Lawyers

Assessing Hidden Risks of LLMs: An Empirical Study on Robustness, Consistency, and Credibility

Large Language Models "Ad Referendum": How Good Are They at Machine Translation in the Legal Domain?

Large Models of What? Mistaking Engineering Achievements for Human Linguistic Agency

Awes, Laws, and Flaws From Today's LLM Research

Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilities

Large language models effectively leverage document-level context for literary translation, but critical errors persist