Abstract:Optimization modeling and solving play a critical role in the application of Operations Research (OR) tools to address real-world problems, yet they pose challenges and require extensive expertise from OR experts. With the advent of large language models (LLMs), new opportunities have emerged to streamline and automate these tasks. However, current research predominantly relies on closed-source LLMs such as GPT-4, along with extensive prompt engineering techniques. This reliance stems from the scarcity of high-quality training datasets for optimization modeling, resulting in elevated costs, prolonged processing times, and privacy concerns. To address these challenges, our work is the first to propose a viable path for training open-source LLMs that are capable of optimization modeling as well as developing and executing solver codes, eventually leading to a superior ability for automating optimization modeling and solving. Particularly, we introduce a semi-automated data synthesis framework designed for optimization modeling issues, named OR-Instruct. This framework merges the training data requirements of large models with the unique characteristics of optimization modeling problems, and allows for customizable enhancements tailored to specific scenarios or modeling types. To evaluate the performance of our proposed framework, we present the IndustryOR benchmark, the inaugural industrial standard for evaluating LLMs in solving practical OR problems. Utilizing data synthesized through OR-Instruct, we train various open-source LLMs with a capacity of 7 billion parameters (dubbed ORLMs). The resulting model demonstrates significantly enhanced optimization modeling capabilities, achieving state-of-the-art performance across the NL4OPT, MAMO, and IndustryOR benchmarks. Our code and data are available at \url{<a class="link-external link-https" href="https://github.com/Cardinal-Operations/ORLM" rel="external noopener nofollow">this https URL</a>}.

OR-Gym: A Reinforcement Learning Library for Operations Research Problems

RLOR: A Flexible Framework of Deep Reinforcement Learning for Operation Research

Implementing Reinforcement Learning Algorithms in Retail Supply Chains with OpenAI Gym Toolkit

Controlgym: Large-Scale Control Environments for Benchmarking Reinforcement Learning Algorithms

Gymnasium: A Standard Interface for Reinforcement Learning Environments

LapGym -- An Open Source Framework for Reinforcement Learning in Robot-Assisted Laparoscopic Surgery

Gym-ANM: Open-source software to leverage reinforcement learning for power system management in research and education

EduGym: An Environment and Notebook Suite for Reinforcement Learning Education

Real World Offline Reinforcement Learning with Realistic Data Source

GreenLight-Gym: A Reinforcement Learning Benchmark Environment for Greenhouse Crop Production Control

ORLM: A Customizable Framework in Training Large Models for Automated Optimization Modeling

CaiRL: A High-Performance Reinforcement Learning Environment Toolkit

OffWorld Gym: open-access physical robotics environment for real-world reinforcement learning benchmark and research

ModelicaGym: Applying Reinforcement Learning to Modelica Models

Intrinsically Motivated Multi-Goal Reinforcement Learning Using Robotics Environment Integrated with OpenAI Gym

SDGym: Low-Code Reinforcement Learning Environments using System Dynamics Models

PC-Gym: Benchmark Environments For Process Control Problems

Deep Policy Iteration with Integer Programming for Inventory Management

RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising

RL-ISLAP: A Reinforcement Learning Framework for Industrial-Scale Linear Assignment Problems at Alipay

A Benchmark Environment Motivated by Industrial Control Problems