Abstract:We propose a method to approximate the solution of online mixed-integer optimization (MIO) problems at very high speed using machine learning. By exploiting the repetitive nature of online optimization, we can greatly speed up the solution time. Our approach encodes the optimal solution into a small amount of information denoted as strategy using the voice of optimization framework. In this way, the core part of the optimization routine becomes a multiclass classification problem that can be solved very quickly. In this work, we extend that framework to real-time and high-speed applications focusing on parametric mixed-integer quadratic optimization. We propose an extremely fast online optimization method consisting of a feedforward neural network evaluation and a linear system solution where the matrix has already been factorized. Therefore, this online approach does not require any solver or iterative algorithm. We show the speed of the proposed method both in terms of total computations required and measured execution time. We estimate the number of floating point operations required to completely recover the optimal solution as a function of the problem dimensions. Compared with state-of-the-art MIO routines, the online running time of our method is very predictable and can be lower than a single matrix factorization time. We benchmark our method against the state-of-the-art solver Gurobi obtaining up to two to three orders of magnitude speedups on examples from fuel cell energy management, sparse portfolio optimization, and motion planning with obstacle avoidance. Summary of Contribution: We propose a technique to approximate the solution of online optimization problems at high speed using machine learning. By exploiting the repetitive nature of online optimization, we learn the mapping between the key problem parameters and an encoding of the optimal solution to greatly speed up the solution time. This allows us to significantly improve the computation time and resources needed to solve online mixed-integer optimization problems. We obtain a simple method with a very low computing time variance, which is crucial in online settings.

Fast Bayesian optimization of Needle-in-a-Haystack problems using zooming memory-based initialization (ZoMBI)

High-dimensional Bayesian optimization using low-dimensional feature spaces

Large-Batch, Iteration-Efficient Neural Bayesian Design Optimization

Speed-up of posterior inference of highly-parameterized environmental models from a Kalman proposal distribution: DREAM(KZS)

SCORE: A 1D Reparameterization Technique to Break Bayesian Optimization's Curse of Dimensionality

BackboneLearn: A Library for Scaling Mixed-Integer Optimization-Based Machine Learning

Diagnosing and fixing common problems in Bayesian optimization for molecule design

Solving 0–1 knapsack problem by a novel binary monarch butterfly optimization

Bayesian Optimization in a Billion Dimensions via Random Embeddings

Non-Myopic Multi-Objective Bayesian Optimization

Accelerating Look-ahead in Bayesian Optimization: Multilevel Monte Carlo is All you Need

An Empirical Evaluation of Zeroth-Order Optimization Methods on AI-driven Molecule Optimization

BEACON: A Bayesian Optimization Strategy for Novelty Search in Expensive Black-Box Systems

Accelerating Black-Box Molecular Property Optimization by Adaptively Learning Sparse Subspaces

Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets

B2Opt: Learning to Optimize Black-box Optimization with Little Budget

Decreasing the Computing Time of Bayesian Optimization using Generalizable Memory Pruning

Accelerating Hyperparameter Optimization of Deep Neural Network via Progressive Multi-Fidelity Evaluation.

Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark

Online Mixed-Integer Optimization in Milliseconds

Prior-mean-assisted Bayesian optimization application on FRIB Front-End tunning