Abstract:Optimization modeling and solving play a critical role in the application of Operations Research (OR) tools to address real-world problems, yet they pose challenges and require extensive expertise from OR experts. With the advent of large language models (LLMs), new opportunities have emerged to streamline and automate these tasks. However, current research predominantly relies on closed-source LLMs such as GPT-4, along with extensive prompt engineering techniques. This reliance stems from the scarcity of high-quality training datasets for optimization modeling, resulting in elevated costs, prolonged processing times, and privacy concerns. To address these challenges, our work is the first to propose a viable path for training open-source LLMs that are capable of optimization modeling as well as developing and executing solver codes, eventually leading to a superior ability for automating optimization modeling and solving. Particularly, we introduce a semi-automated data synthesis framework designed for optimization modeling issues, named OR-Instruct. This framework merges the training data requirements of large models with the unique characteristics of optimization modeling problems, and allows for customizable enhancements tailored to specific scenarios or modeling types. To evaluate the performance of our proposed framework, we present the IndustryOR benchmark, the inaugural industrial standard for evaluating LLMs in solving practical OR problems. Utilizing data synthesized through OR-Instruct, we train various open-source LLMs with a capacity of 7 billion parameters (dubbed ORLMs). The resulting model demonstrates significantly enhanced optimization modeling capabilities, achieving state-of-the-art performance across the NL4OPT, MAMO, and IndustryOR benchmarks. Our code and data are available at \url{<a class="link-external link-https" href="https://github.com/Cardinal-Operations/ORLM" rel="external noopener nofollow">this https URL</a>}.

NL2OR: Solve Complex Operations Research Problems Using Natural Language Inputs

Chain-of-Experts: when LLMs Meet Complex Operations Research Problems

Navigating Complexity: Orchestrated Problem Solving with Multi-Agent LLMs

OptiMUS-0.3: Using Large Language Models to Model and Solve Optimization Problems at Scale

ORLM: A Customizable Framework in Training Large Models for Automated Optimization Modeling

Solving General Natural-Language-Description Optimization Problems with Large Language Models

Optimal Decision Making Through Scenario Simulations Using Large Language Models

nl2spec: Interactively Translating Unstructured Natural Language to Temporal Logics with Large Language Models

OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models

Ner4Opt: named entity recognition for optimization modelling from natural language

Large Language Models for Supply Chain Optimization

Autonomous Multi-Objective Optimization Using Large Language Model

Solving Math Word Problems by Combining Language Models With Symbolic Solvers

Large Language Models for Scholarly Ontology Generation: An Extensive Analysis in the Engineering Field

City-LEO: Toward Transparent City Management Using LLM with End-to-End Optimization

Nl2Hltl2Plan: Scaling Up Natural Language Understanding for Multi-Robots Through Hierarchical Temporal Logic Task Representation

A Survey of NL2SQL with Large Language Models: Where are we, and where are we going?

Large language models for automatic equation discovery of nonlinear dynamics

Beyond LLMs: Advancing the Landscape of Complex Reasoning

Natural Language Processing for Requirements Formalization: How to Derive New Approaches?