Abstract:Optimization modeling and solving play a critical role in the application of Operations Research (OR) tools to address real-world problems, yet they pose challenges and require extensive expertise from OR experts. With the advent of large language models (LLMs), new opportunities have emerged to streamline and automate these tasks. However, current research predominantly relies on closed-source LLMs such as GPT-4, along with extensive prompt engineering techniques. This reliance stems from the scarcity of high-quality training datasets for optimization modeling, resulting in elevated costs, prolonged processing times, and privacy concerns. To address these challenges, our work is the first to propose a viable path for training open-source LLMs that are capable of optimization modeling as well as developing and executing solver codes, eventually leading to a superior ability for automating optimization modeling and solving. Particularly, we introduce a semi-automated data synthesis framework designed for optimization modeling issues, named OR-Instruct. This framework merges the training data requirements of large models with the unique characteristics of optimization modeling problems, and allows for customizable enhancements tailored to specific scenarios or modeling types. To evaluate the performance of our proposed framework, we present the IndustryOR benchmark, the inaugural industrial standard for evaluating LLMs in solving practical OR problems. Utilizing data synthesized through OR-Instruct, we train various open-source LLMs with a capacity of 7 billion parameters (dubbed ORLMs). The resulting model demonstrates significantly enhanced optimization modeling capabilities, achieving state-of-the-art performance across the NL4OPT, MAMO, and IndustryOR benchmarks. Our code and data are available at \url{<a class="link-external link-https" href="https://github.com/Cardinal-Operations/ORLM" rel="external noopener nofollow">this https URL</a>}.

OMLT: Optimization & Machine Learning Toolkit

modOpt: A modular development environment and library for optimization algorithms

The Open MatSci ML Toolkit: A Flexible Framework for Machine Learning in Materials Science

OptiMUS-0.3: Using Large Language Models to Model and Solve Optimization Problems at Scale

OptiMUS: Scalable Optimization Modeling with (MI)LP Solvers and Large Language Models

ENTMOOT: A Framework for Optimization over Ensemble Tree Models

OLMo: Accelerating the Science of Language Models

Learning to optimize: A tutorial for continuous and mixed-integer optimization

Optimization Networks for Integrated Machine Learning

LLMOPT: Learning to Define and Solve General Optimization Problems from Scratch

ODTlearn: A Package for Learning Optimal Decision Trees for Prediction and Prescription

Generalized Optimization Modulo Theories

Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent System

MLMOD: Machine Learning Methods for Data-Driven Modeling in LAMMPS

Robust Optimization Made Easy with ROME

Machine Learning Insides OptVerse AI Solver: Design Principles and Applications

CATO: End-to-End Optimization of ML-Based Traffic Analysis Pipelines

Talaria: Interactively Optimizing Machine Learning Models for Efficient Inference

OML: Open, Monetizable, and Loyal AI

ORLM: A Customizable Framework in Training Large Models for Automated Optimization Modeling

NOMAD version 4: Nonlinear optimization with the MADS algorithm