Stochastic models for online optimization

Umberto Casti,Sandro Zampieri

2024-11-28

Abstract:In this paper, we propose control-theoretic methods as tools for the design of online optimization algorithms that are able to address dynamic, noisy, and partially uncertain time-varying quadratic objective functions. Our approach introduces two algorithms specifically tailored for scenarios where the cost function follows a stochastic linear model. The first algorithm is based on a Kalman filter-inspired approach, leveraging state estimation techniques to account for the presence of noise in the evolution of the objective function. The second algorithm applies $\mathcal{H}_\infty$-robust control strategies to enhance performance under uncertainty, particularly in cases in which model parameters are characterized by a high variability. Through numerical experiments, we demonstrate that our algorithms offer significant performance advantages over the traditional gradient-based method and also over the optimization strategy proposed in <a class="link-https" data-arxiv-id="2205.13932" href="https://arxiv.org/abs/2205.13932">arXiv:2205.13932</a> based on deterministic models.

Optimization and Control,Systems and Control

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the problem of dynamic, noisy, and partially uncertain time - varying quadratic objective functions encountered in online optimization. Specifically, the authors propose two control - theory - based methods to design online optimization algorithms to deal with the situation where the cost function follows a stochastic linear model.

### Problem Background

Online optimization is a rapidly developing field with wide applications, including in areas such as control, signal processing, and machine learning. The importance of these problems increases with technological progress because they involve optimizing time - varying cost functions in dynamic environments. Traditional methods such as gradient descent usually perform well in static environments but are difficult to achieve optimal performance in dynamic environments.

### Research Motivation

Existing methods mainly solve optimization problems in dynamic environments by adapting static methods (such as gradient descent). Although these methods can approach the optimal trajectory, they fail to fully utilize the dynamic characteristics of the problem to improve performance. In contrast, structured algorithms improve tracking performance by taking advantage of the evolution of the cost function and assume the rate of change of the cost function to predict and adjust the solution. However, these methods do not fully utilize a more accurate model of the time evolution of the cost function.

### Paper Contributions

The methods proposed in this paper aim to fill this gap by introducing two new algorithms:
1. **Kalman - filtering - based algorithm**: Utilizes state - estimation techniques to handle the noise in the evolution of the objective function.
2. **H∞ - robust - control - based algorithm**: Enhances performance under uncertainty, especially when the model parameters have high variability.

### Numerical Experiment Results

Through numerical experiments, the authors show that the proposed algorithms significantly outperform the traditional gradient - descent method and other optimization strategies based on deterministic models in terms of performance. The experimental results indicate that, under different levels of measurement noise and different maximum eigenvalues, both new algorithms can provide better performance.

### Summary

This paper solves the problem of dynamic, noisy, and partially uncertain time - varying quadratic objective functions encountered in online optimization, proposes two new algorithms based on control theory, and verifies their superiority through experiments. Future research can extend these methods to a wider range of online optimization problems, relax the quadratic assumption of the cost function, and handle other types of uncertainty.

### Formula Representation

The key formulas involved in the paper are as follows:

- The form of the objective function:
\[
f_k(x)=\frac{1}{2}(x - c_k)^T A(x - c_k)+d_k
\]
where \(A\in\mathbb{R}^{n\times n}\) is a fixed positive - definite matrix, and \(c_k\in\mathbb{R}^n\) and \(d_k\in\mathbb{R}\) are time - varying.

- The gradient expression:
\[
g_k:=\nabla f_k(x_k)=A(x_k - c_k)
\]

- The cost function:
\[
J:=\lim_{k\rightarrow+\infty}E\left[e_k^T e_k\right]
\]
where \(e_k := x_k - c_k\).

- The H2 - norm:
\[
J=\|H_{we}(z)\|_2^2
\]
where \(H_{we}(z)=-(I_n - c(z)A)^{-1}h(z)\).

Through these formulas, the paper details how to design and analyze new online optimization algorithms.