Abstract:Linear-quadratic regulator (LQR) is a landmark problem in the field of optimal control, which is the concern of this paper. Generally, LQR is classified into state-feedback LQR (SLQR) and output-feedback LQR (OLQR) based on whether the full state is obtained. It has been suggested in existing literature that both SLQR and OLQR could be viewed as \textit{constrained nonconvex matrix optimization} problems in which the only variable to be optimized is the feedback gain matrix. In this paper, we introduce a first-order accelerated optimization framework of handling the LQR problem, and give its convergence analysis for the cases of SLQR and OLQR, respectively. Specifically, a Lipschiz Hessian property of LQR performance criterion is presented, which turns out to be a crucial property for the application of modern optimization techniques. For the SLQR problem, a continuous-time hybrid dynamic system is introduced, whose solution trajectory is shown to converge exponentially to the optimal feedback gain with Nesterov-optimal order $1-\frac{1}{\sqrt{\kappa}}$ ($\kappa$ the condition number). Then, the symplectic Euler scheme is utilized to discretize the hybrid dynamic system, and a Nesterov-type method with a restarting rule is proposed that preserves the continuous-time convergence rate, i.e., the discretized algorithm admits the Nesterov-optimal convergence order. For the OLQR problem, a Hessian-free accelerated framework is proposed, which is a two-procedure method consisting of semiconvex function optimization and negative curvature exploitation. In a time $\mathcal{O}(\epsilon^{-7/4}\log(1/\epsilon))$, the method can find an $\epsilon$-stationary point of the performance criterion; this entails that the method improves upon the $\mathcal{O}(\epsilon^{-2})$ complexity of vanilla gradient descent. Moreover, our method provides the second-order guarantee of stationary point.

Positive Realness and Optimality Problem for Rectangular Descriptor Systems

Positive Realness And Optimality Problems For Linear Systems Via Dynamic Compensation

Regularisation And Stabilisation Of Linear Discrete-Time Descriptor Systems

Lmi Approach To Optimal Guaranteed Cost Control For A Class Of Linear Uncertain Discrete Systems

Inverse optimality problem for singularly perturbed systems

Stabilizing Model Predictive Control Scheme for Piecewise Affine Systems with Maximal Positively Invariant Terminal Set

ROBUST QUADRATIC STABILIZATION WITH INTEGRITY FOR DESCRIPTOR SYSTEMS

Multivariable robust digital control: a modified dyadic approach

Optimal H∞ output feedback control for a class of nonlinear systems

Achievable Performance of Decentralized Control Systems

Proper Lumping for Positive Bilinear Control Systems

Direct Optimization Based Compensation Adaptive Robust Control of Nonlinear Systems with State and Input Constraints

Mixed Integer Optimal Control of Systems with Typical Nonlinear Components

Infinite Horizon Stabilization and Linear Quadratic Optimal Control of Descriptor Stochastic Markov Jump Systems

GUARANTEED COST CONTROL FOR A CLASS OF DESCRIPTOR SYSTEMS WITH UNCERTAINTIES

Accelerated Optimization Landscape of Linear-Quadratic Regulator

A successive convex optimization method for bilinear matrix inequality problems and its application to static output‐feedback control

Estimate of the Domain of Attraction for Polynomial Systems Using Bisectional Sum-of-Squares Optimization

Stochastic Linear Quadratic Regulators with Indefinite Control Weight Costs. II

Regularization of Linear Discrete-Time Periodic Descriptor Systems by Derivative and Proportional State Feedback

Optimal Adaptive Control of Linear Stochastic Systems with Quadratic Cost Function