Solving General Natural-Language-Description Optimization Problems with Large Language Models

Jihai Zhang,Wei Wang,Siyan Guo,Li Wang,Fangquan Lin,Cheng Yang,Wotao Yin

2024-07-09

Abstract:Optimization problems seek to find the best solution to an objective under a set of constraints, and have been widely investigated in real-world applications. Modeling and solving optimization problems in a specific domain typically require a combination of domain knowledge, mathematical skills, and programming ability, making it difficult for general users and even domain professionals. In this paper, we propose a novel framework called OptLLM that augments LLMs with external solvers. Specifically, OptLLM accepts user queries in natural language, convert them into mathematical formulations and programming codes, and calls the solvers to calculate the results for decision-making. In addition, OptLLM supports multi-round dialogues to gradually refine the modeling and solving of optimization problems. To illustrate the effectiveness of OptLLM, we provide tutorials on three typical optimization applications and conduct experiments on both prompt-based GPT models and a fine-tuned Qwen model using a large-scale selfdeveloped optimization dataset. Experimental results show that OptLLM works with various LLMs, and the fine-tuned model achieves an accuracy boost compared to the promptbased models. Some features of OptLLM framework have been available for trial since June 2023 (<a class="link-external link-https" href="https://opt.alibabacloud.com/chat" rel="external noopener nofollow">this https URL</a> or <a class="link-external link-https" href="https://opt.aliyun.com/chat" rel="external noopener nofollow">this https URL</a>).

Optimization and Control,Artificial Intelligence,Computation and Language,Machine Learning

What problem does this paper attempt to address?

This paper proposes a new framework called OptLLM to address optimization problems described in natural language. Currently, modeling and solving optimization problems often require expertise in mathematics, programming skills, and domain-specific knowledge, which poses challenges for both ordinary users and professionals. OptLLM combines large language models (LLMs) with external solvers to convert user queries expressed in natural language into mathematical formulas and programming code, and then calls the solver to obtain decision results. In addition, OptLLM supports multi-turn dialogues to iteratively refine the modeling and solving of optimization problems.

In the experiments, the researchers demonstrated tutorials of OptLLM in three typical applications and tested it on a large-scale self-built optimization dataset using prompt-based GPT models and fine-tuned Qwen models. The results showed that the fine-tuned models achieved higher accuracy improvements compared to the prompt-based models. Some functionalities of OptLLM have been made available for trial use since June 2023.

The paper also discusses the limitations of LLMs in arithmetic and logical reasoning, as well as data privacy issues in online services. OptLLM addresses these issues by utilizing open-source LLMs or online LLM services. However, it may be constrained by the knowledge base limitations of these LLMs (e.g., only updated until 2023) when dealing with incomplete problems that require additional information.

In conclusion, the goal of OptLLM is to simplify the modeling and solving process of optimization problems, making it more accessible for non-experts to make decisions. Future work will focus on increasing the diversity of optimization problems, improving the arithmetic and logical reasoning capabilities of the model, and exploring more open-source LLMs and evaluation methods.