Abstract:Linear-quadratic regulator (LQR) is a landmark problem in the field of optimal control, which is the concern of this paper. Generally, LQR is classified into state-feedback LQR (SLQR) and output-feedback LQR (OLQR) based on whether the full state is obtained. It has been suggested in existing literature that both SLQR and OLQR could be viewed as \textit{constrained nonconvex matrix optimization} problems in which the only variable to be optimized is the feedback gain matrix. In this paper, we introduce a first-order accelerated optimization framework of handling the LQR problem, and give its convergence analysis for the cases of SLQR and OLQR, respectively. Specifically, a Lipschiz Hessian property of LQR performance criterion is presented, which turns out to be a crucial property for the application of modern optimization techniques. For the SLQR problem, a continuous-time hybrid dynamic system is introduced, whose solution trajectory is shown to converge exponentially to the optimal feedback gain with Nesterov-optimal order $1-\frac{1}{\sqrt{\kappa}}$ ($\kappa$ the condition number). Then, the symplectic Euler scheme is utilized to discretize the hybrid dynamic system, and a Nesterov-type method with a restarting rule is proposed that preserves the continuous-time convergence rate, i.e., the discretized algorithm admits the Nesterov-optimal convergence order. For the OLQR problem, a Hessian-free accelerated framework is proposed, which is a two-procedure method consisting of semiconvex function optimization and negative curvature exploitation. In a time $\mathcal{O}(\epsilon^{-7/4}\log(1/\epsilon))$, the method can find an $\epsilon$-stationary point of the performance criterion; this entails that the method improves upon the $\mathcal{O}(\epsilon^{-2})$ complexity of vanilla gradient descent. Moreover, our method provides the second-order guarantee of stationary point.

Analysis of linear‐quadratic optimal control via a hybrid control lens

Digital LQR Steady-State Optimal Control with Feedforward for Nonminimum Phase Boost DC-DC Converter

A Digital Dual-Mode Lqr Optimal Controller for Boost Converter over A Wide Operation Range

Optimal Control for Hybrid Systems Based on Mixed Dynamic Programming

Dual-Mode Lqr-Feedforward Optimal Control for Non-Minimum Phase Boost Converter

Improved Control Using PFC and Modified Linear Quadratic Regulator for a Kind of Nonlinear Systems

Accelerated Optimization Landscape of Linear-Quadratic Regulator

On the Optimization Landscape of Dynamic Output Feedback Linear Quadratic Control

Linear Quadratic Optimal Control for Systems Governed by First-Order Hyperbolic Partial Differential Equations

Optimal Nonlinear Feedback Control of Quasi-Hamiltonian Systems

Stochastic Optimal Linear Quadratic Regulation Control of Discrete-time Systems with Delay and Quadratic Constraints

A Hybrid Optimization Approach for Discrete-Time H<inf>&#x221E;</inf> Static-Output-feedback Control Problem

Linear-Quadratic Optimal Control in Maximal Coordinates

Analysis of the Optimization Landscape of Linear Quadratic Gaussian (LQG) Control

An LQR‐Lagrange Algorithm Generated by Interdisciplinary Integration With Optimal Control and Optimization

LQ Optimal Control of First-Order Hyperbolic PDE Systems with Final State Constraints

On Stochastic Optimal Control of Partially Observable Nonlinear Quasi Hamiltonian Systems

Optimal Control of State-Triggered Linear Hybrid Systems

Adaptive Optimal Control with Guaranteed Convergence Rate for Continuous-Time Linear Systems with Completely Unknown Dynamics.

Revisiting LQR Control from the Perspective of Receding-Horizon Policy Gradient