Abstract:Linear-quadratic regulator (LQR) is a landmark problem in the field of optimal control, which is the concern of this paper. Generally, LQR is classified into state-feedback LQR (SLQR) and output-feedback LQR (OLQR) based on whether the full state is obtained. It has been suggested in existing literature that both SLQR and OLQR could be viewed as \textit{constrained nonconvex matrix optimization} problems in which the only variable to be optimized is the feedback gain matrix. In this paper, we introduce a first-order accelerated optimization framework of handling the LQR problem, and give its convergence analysis for the cases of SLQR and OLQR, respectively. Specifically, a Lipschiz Hessian property of LQR performance criterion is presented, which turns out to be a crucial property for the application of modern optimization techniques. For the SLQR problem, a continuous-time hybrid dynamic system is introduced, whose solution trajectory is shown to converge exponentially to the optimal feedback gain with Nesterov-optimal order $1-\frac{1}{\sqrt{\kappa}}$ ($\kappa$ the condition number). Then, the symplectic Euler scheme is utilized to discretize the hybrid dynamic system, and a Nesterov-type method with a restarting rule is proposed that preserves the continuous-time convergence rate, i.e., the discretized algorithm admits the Nesterov-optimal convergence order. For the OLQR problem, a Hessian-free accelerated framework is proposed, which is a two-procedure method consisting of semiconvex function optimization and negative curvature exploitation. In a time $\mathcal{O}(\epsilon^{-7/4}\log(1/\epsilon))$, the method can find an $\epsilon$-stationary point of the performance criterion; this entails that the method improves upon the $\mathcal{O}(\epsilon^{-2})$ complexity of vanilla gradient descent. Moreover, our method provides the second-order guarantee of stationary point.

Constrained LQR Design Using Interior-Point Arc-Search Method for Convex Quadratic Programming with Box Constraints

Solving Convex Quadratic Programming by Potential-Reduction Interior-Point Algorithm

An Efficient Interior-Point Algorithm with New Non-Monotone Line Search Filter Method for Nonlinear Constrained Programming

An Off-line Algorithm of Constrained Predictive Control Model Using Linear Matrix Inequalities

Time-Optimal Constrained Adaptive Robust Control of Single-DOF Mechanical Systems: A Comparative Study with BLF-based Methods

Efficient Constrained Dynamics Algorithms Based on an Equivalent LQR Formulation Using Gauss' Principle of Least Constraint

An Infeasible Interior-point Arc-search Algorithm for Nonlinear Constrained Optimization

Parallel and Proximal Constrained Linear-Quadratic Methods for Real-Time Nonlinear MPC

A Design Approach of Constrained Linear Time-Varying Quadratic Regulation

On Linear Quadratic Optimal Control of Discrete-Time Complex-Valued Linear Systems

Design and analysis of constrained nonlinear quadratic regulator.

A Novel Approach to Constrained Generic Model Control Based on Quadratic Programming

A computationally efficient arc-search interior-point algorithm for nonlinear constrained optimization

An Accelerated Proximal Gradient-Based Algorithm for Quadratic Programming

Learning Algorithm for LQG Model with Constrained Control

Accelerated Optimization Landscape of Linear-Quadratic Regulator

Model-free design of stochastic LQR controller from a primal–dual optimization perspective

The Quadratic-Quadratic Regulator Problem: Approximating feedback controls for quadratic-in-state nonlinear systems

An LQR‐Lagrange Algorithm Generated by Interdisciplinary Integration With Optimal Control and Optimization

Design of Suboptimal Constrained Nonlinear Quadratic Regulator Via Invariant Sets Switching