Abstract:Deep learning has had a far reaching impact in robotics. Specifically, deep reinforcement learning algorithms have been highly effective in synthesizing neural-network controllers for a wide range of tasks. However, despite this empirical success, these controllers still lack theoretical guarantees on their performance, such as Lyapunov stability (i.e., all trajectories of the closed-loop system are guaranteed to converge to a goal state under the control policy). This is in stark contrast to traditional model-based controller design, where principled approaches (like LQR) can synthesize stable controllers with provable guarantees. To address this gap, we propose a generic method to synthesize a Lyapunov-stable neural-network controller, together with a neural-network Lyapunov function to simultaneously certify its stability. Our approach formulates the Lyapunov condition verification as a mixed-integer linear program (MIP). Our MIP verifier either certifies the Lyapunov condition, or generates counter examples that can help improve the candidate controller and the Lyapunov function. We also present an optimization program to compute an inner approximation of the region of attraction for the closed-loop system. We apply our approach to robots including an inverted pendulum, a 2D and a 3D quadrotor, and showcase that our neural-network controller outperforms a baseline LQR controller. The code is open sourced at \url{https://github.com/StanfordASL/neural-network-lyapunov}.

Stability Margins of Neural Network Controllers

Robust Stabilization for a Class of Nonlinear Networked Control Systems

Stability Analysis Using Quadratic Constraints for Systems With Neural Network Controllers

Synthesizing Neural Network Controllers with Closed-Loop Dissipativity Guarantees

New Approach to Stochastic Stability and Controller Design for Networked Control Systems

A Conformal Mapping and Interpolation Approach to Delay Margin for General Unstable Plants

Adaptive Neural-Network Boundary Control for a Flexible Manipulator with Input Constraints and Model Uncertainties

Neural Network Optimal Feedback Control with Guaranteed Local Stability

Robust Stability of Neural Network-controlled Nonlinear Systems with Parametric Variability

Lyapunov-stable neural-network control

Imitation Learning with Stability and Safety Guarantees

Robust Stability of Neural Feedback Systems with Interval Matrix Uncertainties

A Stability Based Neural Networks Controller Design Method

Validation of Neural Network Controllers for Uncertain Systems Through Keep-Close Approach: Robustness Analysis and Safety Verification

Neural network optimal feedback control with enhanced closed loop stability

Stability Margins of $\mathcal{L}_1$ Adaptive Controller: Part II

Data-Driven Stable Neural Feedback Loop Design

Classical Stability Margins by PID Control

Model-Free Verification for Neural Network Controlled Systems

Neural Network Algorithm for Stabilizing Control of Robustness in Systems

Optimal distributed control with stability guarantees by training a network of neural closed-loop maps