Abstract:Linear-quadratic regulator (LQR) is a landmark problem in the field of optimal control, which is the concern of this paper. Generally, LQR is classified into state-feedback LQR (SLQR) and output-feedback LQR (OLQR) based on whether the full state is obtained. It has been suggested in existing literature that both SLQR and OLQR could be viewed as \textit{constrained nonconvex matrix optimization} problems in which the only variable to be optimized is the feedback gain matrix. In this paper, we introduce a first-order accelerated optimization framework of handling the LQR problem, and give its convergence analysis for the cases of SLQR and OLQR, respectively. Specifically, a Lipschiz Hessian property of LQR performance criterion is presented, which turns out to be a crucial property for the application of modern optimization techniques. For the SLQR problem, a continuous-time hybrid dynamic system is introduced, whose solution trajectory is shown to converge exponentially to the optimal feedback gain with Nesterov-optimal order $1-\frac{1}{\sqrt{\kappa}}$ ($\kappa$ the condition number). Then, the symplectic Euler scheme is utilized to discretize the hybrid dynamic system, and a Nesterov-type method with a restarting rule is proposed that preserves the continuous-time convergence rate, i.e., the discretized algorithm admits the Nesterov-optimal convergence order. For the OLQR problem, a Hessian-free accelerated framework is proposed, which is a two-procedure method consisting of semiconvex function optimization and negative curvature exploitation. In a time $\mathcal{O}(\epsilon^{-7/4}\log(1/\epsilon))$, the method can find an $\epsilon$-stationary point of the performance criterion; this entails that the method improves upon the $\mathcal{O}(\epsilon^{-2})$ complexity of vanilla gradient descent. Moreover, our method provides the second-order guarantee of stationary point.

Revisiting the LQR Problem of Singular Systems

Singular LQ Sub-Optimal Control Problem for a Class of Discrete-Time Singular Systems with Time-Delay

The Quadratic-Quadratic Regulator Problem: Approximating feedback controls for quadratic-in-state nonlinear systems

Accelerated Optimization Landscape of Linear-Quadratic Regulator

From LQR to Static Output Feedback: a New LMI Approach

Inverse Value Iteration and Q -Learning: Algorithms, Stability, and Robustness

On the Certainty-Equivalence Approach to Direct Data-Driven LQR Design

Single Time-scale Actor-critic Method to Solve the Linear Quadratic Regulator with Convergence Guarantees

Structured Policy Iteration for Linear Quadratic Regulator

Multi-Objective LQR with Linear Scalarization

i2LQR: Iterative LQR for Iterative Tasks in Dynamic Environments

Data-Driven LQR using Reinforcement Learning and Quadratic Neural Networks

Infinite-horizon Risk-constrained Linear Quadratic Regulator with Average Cost

Distributed Cooperative LQR Design for Multi-Input Linear Systems

An efficient algorithm for solving linear equality-constrained LQR problems

Direct Data-Driven Discounted Infinite Horizon Linear Quadratic Regulator with Robustness Guarantees

Stochastic Linear Quadratic Regulators with Indefinite Control Weight Costs. II

Efficient Constrained Dynamics Algorithms Based on an Equivalent LQR Formulation Using Gauss' Principle of Least Constraint

Linear-Quadratic Problems in Systems and Controls via Covariance Representations and Linear-Conic Duality: Finite-Horizon Case

The Power of Linear Controllers in LQR Control