Abstract:Linear-quadratic regulator (LQR) is a landmark problem in the field of optimal control, which is the concern of this paper. Generally, LQR is classified into state-feedback LQR (SLQR) and output-feedback LQR (OLQR) based on whether the full state is obtained. It has been suggested in existing literature that both SLQR and OLQR could be viewed as \textit{constrained nonconvex matrix optimization} problems in which the only variable to be optimized is the feedback gain matrix. In this paper, we introduce a first-order accelerated optimization framework of handling the LQR problem, and give its convergence analysis for the cases of SLQR and OLQR, respectively. Specifically, a Lipschiz Hessian property of LQR performance criterion is presented, which turns out to be a crucial property for the application of modern optimization techniques. For the SLQR problem, a continuous-time hybrid dynamic system is introduced, whose solution trajectory is shown to converge exponentially to the optimal feedback gain with Nesterov-optimal order $1-\frac{1}{\sqrt{\kappa}}$ ($\kappa$ the condition number). Then, the symplectic Euler scheme is utilized to discretize the hybrid dynamic system, and a Nesterov-type method with a restarting rule is proposed that preserves the continuous-time convergence rate, i.e., the discretized algorithm admits the Nesterov-optimal convergence order. For the OLQR problem, a Hessian-free accelerated framework is proposed, which is a two-procedure method consisting of semiconvex function optimization and negative curvature exploitation. In a time $\mathcal{O}(\epsilon^{-7/4}\log(1/\epsilon))$, the method can find an $\epsilon$-stationary point of the performance criterion; this entails that the method improves upon the $\mathcal{O}(\epsilon^{-2})$ complexity of vanilla gradient descent. Moreover, our method provides the second-order guarantee of stationary point.

Evaluation of Input Redundancies on Linear Quadratic Regulator Problems

The Effects of Adding Input Redundancies in Linear Quadratic Regulator Problems

Design of Robust Regulators for Nonlinear Systems with A Class of Unmatched Uncertainties

A unified study of the redundant control input problem with different conditions in the stabilizable case

On the Effects of Redundant Control Inputs.

Improved Dynamic Output Feedback Rmpc for Linear Uncertain Systems with Input Constraints

Reduction of data amount in data-driven design of linear quadratic regulators

Pre-processing Nonlinear Output Regulation with Non-Vanishing Measurements

Sparsity-Constrained Linear Quadratic Regulation Problem: Greedy Approach with Performance Guarantee

The Effects of Redundant Control Inputs in Optimal Control

Reduction in Optimal Time in Systems with Input Redundancy

Robustness for uncertain linear quadratic regulator systems

Output Regulation of Multivariable Nonlinear Systems Using Extra Outputs

Robust Regulator Design of General Linear Systems with Sampled Measurements

Guaranteed Stability Margins for Decentralized Linear Quadratic Regulators

Robust implementable regulator design of linear systems with non-vanishing measurements

Robust Output Regulation for a Class of Nonlinear Systems Not Detectable by the Regulated Output

The Fundamental Limitations of Learning Linear-Quadratic Regulators

Robustness bounds for continuous systems with LQ regulators

Accelerated Optimization Landscape of Linear-Quadratic Regulator

Rate-matching the regret lower-bound in the linear quadratic regulator with unknown dynamics