Abstract:Linear-quadratic regulator (LQR) is a landmark problem in the field of optimal control, which is the concern of this paper. Generally, LQR is classified into state-feedback LQR (SLQR) and output-feedback LQR (OLQR) based on whether the full state is obtained. It has been suggested in existing literature that both SLQR and OLQR could be viewed as \textit{constrained nonconvex matrix optimization} problems in which the only variable to be optimized is the feedback gain matrix. In this paper, we introduce a first-order accelerated optimization framework of handling the LQR problem, and give its convergence analysis for the cases of SLQR and OLQR, respectively. Specifically, a Lipschiz Hessian property of LQR performance criterion is presented, which turns out to be a crucial property for the application of modern optimization techniques. For the SLQR problem, a continuous-time hybrid dynamic system is introduced, whose solution trajectory is shown to converge exponentially to the optimal feedback gain with Nesterov-optimal order $1-\frac{1}{\sqrt{\kappa}}$ ($\kappa$ the condition number). Then, the symplectic Euler scheme is utilized to discretize the hybrid dynamic system, and a Nesterov-type method with a restarting rule is proposed that preserves the continuous-time convergence rate, i.e., the discretized algorithm admits the Nesterov-optimal convergence order. For the OLQR problem, a Hessian-free accelerated framework is proposed, which is a two-procedure method consisting of semiconvex function optimization and negative curvature exploitation. In a time $\mathcal{O}(\epsilon^{-7/4}\log(1/\epsilon))$, the method can find an $\epsilon$-stationary point of the performance criterion; this entails that the method improves upon the $\mathcal{O}(\epsilon^{-2})$ complexity of vanilla gradient descent. Moreover, our method provides the second-order guarantee of stationary point.

General Lyapunov-Based Iterative Algorithm for Linear Quadratic Regulator Problem of Stochastic Systems with Markovian Jump

Design of Robust Regulators for Nonlinear Systems with A Class of Unmatched Uncertainties

Accelerated Optimization Landscape of Linear-Quadratic Regulator

Indefinite Robust Linear Quadratic Optimal Regulator for Discrete-Time Uncertain Singular Markov Jump Systems

Infinite-horizon Risk-constrained Linear Quadratic Regulator with Average Cost

A Successive over Relaxation Implicit Iterative Algorithm for Solving Stochastic Linear Systems with Markov Jumps

Modified Riccati iterative algorithms for stochastic coupled algebraic Riccati equations of linear stochastic Markovian jump systems

Cooperative Output Regulation Quadratic Control for Discrete-Time Heterogeneous Multiagent Markov Jump Systems

Linear Quadratic Regulation and Stabilization of Discrete-Time Systems With Delay and Multiplicative Noise

Revisiting the LQR Problem of Singular Systems

Single Time-scale Actor-critic Method to Solve the Linear Quadratic Regulator with Convergence Guarantees

Convergence and Robustness of Value and Policy Iteration for the Linear Quadratic Regulator

The Quadratic-Quadratic Regulator Problem: Approximating feedback controls for quadratic-in-state nonlinear systems

Guaranteed Control Performance Robust LQG Regulator for Discrete-Time Markovian Jump Systems with Uncertain Noise

A model-free first-order method for linear quadratic regulator with $\tilde{O}(1/\varepsilon)$ sampling complexity

Exponentially Stable Adaptive Optimal Control of Uncertain LTI Systems

Stochastic Linear Quadratic Regulators with Indefinite Control Weight Costs. II

Iterative Solutions of Coupled Discrete Markovian Jump Lyapunov Equations

Direct Data-Driven Discounted Infinite Horizon Linear Quadratic Regulator with Robustness Guarantees

Robust JLQG regulator of guaranteed control performance with uncertain noise