Abstract:Linear-quadratic regulator (LQR) is a landmark problem in the field of optimal control, which is the concern of this paper. Generally, LQR is classified into state-feedback LQR (SLQR) and output-feedback LQR (OLQR) based on whether the full state is obtained. It has been suggested in existing literature that both SLQR and OLQR could be viewed as \textit{constrained nonconvex matrix optimization} problems in which the only variable to be optimized is the feedback gain matrix. In this paper, we introduce a first-order accelerated optimization framework of handling the LQR problem, and give its convergence analysis for the cases of SLQR and OLQR, respectively. Specifically, a Lipschiz Hessian property of LQR performance criterion is presented, which turns out to be a crucial property for the application of modern optimization techniques. For the SLQR problem, a continuous-time hybrid dynamic system is introduced, whose solution trajectory is shown to converge exponentially to the optimal feedback gain with Nesterov-optimal order $1-\frac{1}{\sqrt{\kappa}}$ ($\kappa$ the condition number). Then, the symplectic Euler scheme is utilized to discretize the hybrid dynamic system, and a Nesterov-type method with a restarting rule is proposed that preserves the continuous-time convergence rate, i.e., the discretized algorithm admits the Nesterov-optimal convergence order. For the OLQR problem, a Hessian-free accelerated framework is proposed, which is a two-procedure method consisting of semiconvex function optimization and negative curvature exploitation. In a time $\mathcal{O}(\epsilon^{-7/4}\log(1/\epsilon))$, the method can find an $\epsilon$-stationary point of the performance criterion; this entails that the method improves upon the $\mathcal{O}(\epsilon^{-2})$ complexity of vanilla gradient descent. Moreover, our method provides the second-order guarantee of stationary point.

Positive Realness And Optimality Problems For Linear Systems Via Dynamic Compensation

Positive Realness and Optimality Problem for Rectangular Descriptor Systems

Dynamic Output Feedback Stabilizing Control For Linear Time-Varying Uncertain Systems With Delayed State And Control

Inverse optimality problem for singularly perturbed systems

An Off-line Algorithm of Constrained Predictive Control Model Using Linear Matrix Inequalities

Stabilizing Model Predictive Control Scheme for Piecewise Affine Systems with Maximal Positively Invariant Terminal Set

A Learning-Based Optimal Tracking Controller for Continuous Linear Systems with Unknown Dynamics: Theory and Case Study

Direct Optimization Based Compensation Adaptive Robust Control of Nonlinear Systems with State and Input Constraints

A successive convex optimization method for bilinear matrix inequality problems and its application to static output‐feedback control

Optimal H∞ output feedback control for a class of nonlinear systems

Local Stabilizing Control for Polynomial Nonlinear Systems

Weak Closed-loop Solvability of Linear Quadratic Stochastic Optimal Control Problems with Partial Information

Precision Motion Control of Constrained SISO Nonlinear System Via Direct Optimized Compensation

Linear-Quadratic Problems in Systems and Controls via Covariance Representations and Linear-Conic Duality: Finite-Horizon Case

On the convexity of static output feedback control synthesis for systems with lossless nonlinearities

Accelerated Optimization Landscape of Linear-Quadratic Regulator

Optimal Adaptive Control of Linear Stochastic Systems with Quadratic Cost Function

LQ Optimal Control of First-Order Hyperbolic PDE Systems with Final State Constraints

Discrete Time Mean-Field Stochastic Linear-Quadratic Optimal Control Problems

Proper Lumping for Positive Bilinear Control Systems

Stochastic Linear-Quadratic Control with Conic Control Constraints on an Infinite Time Horizon