Abstract:The linear quadratic regulator (LQR) problem is a cornerstone of automatic control, and it has been widely studied in the data-driven setting. The various data-driven approaches can be classified as indirect (i.e., based on an identified model) versus direct or as robust (i.e., taking uncertainty into account) versus certainty-equivalence. Here we show how to bridge these different formulations and propose a novel, direct, and regularized formulation. We start from indirect certainty-equivalence LQR, i.e., least-square identification of state-space matrices followed by a nominal model-based design, formalized as a bi-level program. We show how to transform this problem into a single-level, regularized, and direct data-driven control formulation, where the regularizer accounts for the least-square data fitting criterion. For this novel formulation we carry out a robustness and performance analysis in presence of noisy data. Our proposed direct and regularized formulation is also amenable to be further blended with a robust-stability-promoting regularizer. In a numerical case study we compare regularizers promoting either robustness or certainty-equivalence, and we demonstrate the remarkable performance when blending both of them.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is several key challenges in data - driven control design of the Linear Quadratic Regulator (LQR). Specifically, the paper focuses on how to achieve LQR control design through data - driven methods when dealing with Linear Time - Invariant (LTI) systems with process noise. The main contributions of the paper are as follows:
1. **Bridging different methods**: The paper shows how to combine different data - driven methods (indirect methods and direct methods, robust methods and certainty - equivalence methods) and proposes a new direct and regularized data - driven LQR design method.
2. **From bi - level to single - level optimization**: Starting from the indirect certainty - equivalence LQR, that is, first identifying the state - space matrices by the least - squares method and then designing based on the nominal model, the paper formulates it as a bi - level optimization problem. Then, the paper shows how to transform this problem into a single - level, regularized direct data - driven control problem, where the regularization term takes into account the least - squares data - fitting criterion.
3. **Robustness and performance analysis**: For the newly proposed direct and regularized data - driven control method, the paper conducts robustness and performance analysis, especially in the presence of noisy data. The paper also discusses how to further combine robust stability to promote the regularization term.
4. **Numerical case studies**: In numerical case studies, the paper compares different regularization terms that promote robustness and certainty - equivalence and shows a significant performance improvement when the two are combined.
### Specific problem analysis
#### 1. **Indirect methods and direct methods**
- **Indirect methods**: First, identify the parametric model of the system from the data, and then design the controller based on this model. The advantage of this method is that it can utilize mature control theory tools, but the disadvantage is that it is difficult to propagate uncertainty estimates during the system identification process.
- **Direct methods**: Design the controller directly from the data without explicitly identifying the system model. This method is more sensitive to inaccurate data and needs to be robustified to improve performance.
#### 2. **Certainty - equivalence and robustness**
- **Certainty - equivalence**: Assume that the model parameters are known and error - free, and directly use these parameters for control design. This method is simple and effective, but may perform poorly in the presence of uncertainties and noise.
- **Robustness**: Consider the uncertainties of model parameters and design a controller that can deal with these uncertainties. This method performs better in the case of severe uncertainties, but may not be as good as the certainty - equivalence method in the case of small uncertainties.
#### 3. **New direct and regularized methods**
- **Single - level optimization**: The paper proposes a new single - level optimization problem, which transforms the bi - level optimization problem in the indirect method into a single - level optimization problem containing a regularization term.
- **Regularization term**: The regularization term is used to penalize data - fitting errors, so as to consider the uncertainty of data when designing the controller.
#### 4. **Robustness and performance analysis**
- **Signal - to - noise ratio (SNR)**: The paper defines the signal - to - noise ratio (SNR) and analyzes the stability and performance of the controller under different SNRs.
- **Non - asymptotic analysis**: The paper provides non - asymptotic analysis results, indicating that when the SNR is large enough, the certainty - equivalence method can ensure the stability of the closed - loop system and near - optimal performance.
#### 5. **Numerical case studies**
- **Comparing different methods**: Through numerical experiments, the paper compares the effects of different regularization terms and shows that when the certainty - equivalence and robustness regularization terms are combined, the performance of the controller is significantly improved.
In general, the paper aims to solve the shortcomings of existing methods in terms of robustness and performance, especially in the presence of noisy data, by proposing a new direct and regularized data - driven LQR design method.