Abstract:The subject of the research is machine learning models and methods for generating legal contracts with limited resources and performance evaluation benchmarks. The goal of the work is to analyse approaches of domain-specific Large Language Models development and to find the optimal method of creating independent specialized systems that can generate contracts in different languages and legal systems. The article addresses the following tasks: identification of existing companies and solutions in this area, exploring approaches to create texts in natural language, analysis of evaluation and comparison methods of such systems, inspecting limitations and shortcomings of existing solutions and approaches, finding the optimal method of developing systems with limited resources. The following results were obtained: approaches of natural language generation and their features were investigated; the "Transformer" architecture was defined as a modern standard in the field of text information generation; different model types which are based on this architecture were considered; data sources for training were analysed; methods of adapting models in specialized areas were considered; model evaluating benchmarks for various tasks were reviewed; shortcomings of the existing specialized language models and the incompleteness of existing benchmarks for contract generation task evaluation were revealed. As a result of the analytical experiment, it was determined that the Retrieval-Augmented Generation method is the most optimal for solving the given task under the given conditions. The conducted experiment and its results can be used as a basis for further research of domain-specific language models development with limited resources. Conclusions: the article provides an overview of natural language generation methods using modern machine learning techniques, considers their advantages and disadvantages for small companies and scientific institutions that have limited resources. The work examinates a specialized legal domain and the problem of contract generation and determines the most optimal method to solve it.

AutoMTLSpec: Learning to Generate MTL Specifications from Natural Language Contracts

Automated Generation of LTL Specifications for Smart Home IoT Using Natural Language

TR2MTL: LLM based framework for Metric Temporal Logic Formalization of Traffic Rules

nl2spec: Interactively Translating Unstructured Natural Language to Temporal Logics with Large Language Models

DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications

SpecGen: Automated Generation of Formal Program Specifications via Large Language Models

Formal Specifications from Natural Language

Specification Mining for Smart Contracts with Automatic Abstraction Tuning

Impact of Large Language Models on Generating Software Specifications

Learning to Estimate System Specifications in Linear Temporal Logic using Transformers and Mamba

Enhancing Contract Negotiations with LLM-Based Legal Document Comparison

SpecTra: Enhancing the Code Translation Ability of Language Models by Generating Multi-Modal Specifications

Analysis of methods for training domain-specific language models in the area of legal contracts generation

From Legal Contracts to Formal Specifications: A Systematic Literature Review

Towards the LLM-Based Generation of Formal Specifications from Natural-Language Contracts: Early Experiments with Symboleo

DeepSTL -- From English Requirements to Signal Temporal Logic

TSpec-LLM: An Open-source Dataset for LLM Understanding of 3GPP Specifications

Synthesizing Efficiently Monitorable Formulas in Metric Temporal Logic

TAT-LLM: A Specialized Language Model for Discrete Reasoning over Tabular and Textual Data

Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environments

Towards machine-readable semantic-based E-business contract representations using Network of Timed Automata (NTA)