Learning to optimize: A tutorial for continuous and mixed-integer optimization

Xiaohan Chen,Jialin Liu,Wotao Yin

2024-05-24

Abstract:Learning to Optimize (L2O) stands at the intersection of traditional optimization and machine learning, utilizing the capabilities of machine learning to enhance conventional optimization techniques. As real-world optimization problems frequently share common structures, L2O provides a tool to exploit these structures for better or faster solutions. This tutorial dives deep into L2O techniques, introducing how to accelerate optimization algorithms, promptly estimate the solutions, or even reshape the optimization problem itself, making it more adaptive to real-world applications. By considering the prerequisites for successful applications of L2O and the structure of the optimization problems at hand, this tutorial provides a comprehensive guide for practitioners and researchers alike.

Optimization and Control,Machine Learning

What problem does this paper attempt to address?

### Problems Addressed by the Paper

This paper primarily explores the emerging field of "Learning to Optimize (L2O)." Specifically:

1. **Combining Traditional Optimization with Machine Learning**:
- Traditional optimization methods often need to leverage the structure within data to accelerate or improve the optimization process when dealing with real-world problems. L2O enhances traditional optimization techniques by utilizing the capabilities of machine learning to learn common structural patterns from a large number of instances, thereby achieving better optimization results.

2. **Application Scenarios**:
- When facing repeatedly occurring similar optimization problems (Scenario 1), or situations where it is difficult to directly establish an optimization model (Scenario 2), L2O can find "shortcuts" to the solutions of optimization problems by learning from historical experiences.

3. **Training and Deployment**:
- Unlike traditional optimization methods, L2O methods typically involve two stages: training and deployment. During the training process, a large amount of data is used to improve the optimization algorithm or problem structure; the optimized algorithm obtained after training demonstrates faster speed or higher quality in practical applications.

4. **Three Paradigms**:
- (1) Accelerating the optimization process: Replacing certain traditional steps with machine learning models to speed up traditional algorithms;
- (2) Generating optimization solutions: Directly using machine learning models to generate solutions to optimization problems;
- (3) Adaptive optimization: Adjusting the optimization problem itself through machine learning to make it more aligned with actual needs.

### Summary

This paper aims to introduce how machine learning techniques can be used to improve traditional optimization methods and discusses in detail the three main paradigms of L2O and their application scenarios. Through these methods, the structure within data can be better utilized, thereby improving the speed and quality of the optimization process.