Abstract:PDF HTML XML Export Cite reminder Safe Reinforcement Learning Algorithm and Its Application in Intelligent Control for CPS DOI: 10.21655/ijsi.1673-7288.00284 Author: Affiliation: Clc Number: Fund Project: Article | Figures | Metrics | Reference | Related | Cited by | Materials | Comments Abstract:The design of a safe controller for a Cyber-Physical System (CPS) is a hot research topic. The existing safety controller design based on formal methods has problems such as excessive reliance on models and poor scalability. Intelligent control based on Deep Reinforcement Learning (DRL) can handle high-dimensional nonlinear complex systems and uncertain systems and is becoming a very promising CPS control technology, but it lacks safety guarantees. This study addresses the safety issues of Reinforcement Learning (RL) control by analyzing a typical case of an industrial oil pump control system and carries out research on a Safe Reinforcement Learning (SRL) algorithm and intelligent control application. First, the SRL issue of the industrial oil pump control system is formalized, and a simulation environment of the oil pump is built. Then, by designing the structure and activation function of the output layer, an oil pump controller in the form of a neural network is constructed to satisfy the linear inequality constraints of the on-off operations of the oil pump. Finally, in order to better balance the safety and optimality control objectives, a new SRL algorithm is designed based on the Augmented Lagrange Multiplier (ALM) method. A comparative experiment on the industrial oil pump shows that the controller synthesized by the proposed algorithm surpasses existing similar algorithms both in safety and optimality. During the evaluation, the neural network controllers synthesized in this study pass rigorous formal verification with a probability of 90%. Meanwhile, compared with the theoretically optimal controller, neural network controllers achieve an optimal objective value loss of 2%. The proposed method is expected to be applied in more scenarios, and the case study scheme may provide a reference for other researchers in the field of safe intelligent control and formal verification. Reference Related Cited by

Robust Safe Reinforcement Learning Control of Unknown Continuous-Time Nonlinear Systems with State Constraints and Disturbances

Optimal Control for Constrained Discrete-Time Nonlinear Systems Based on Safe Reinforcement Learning.

Robust Control of Uncertain Linear Systems Based on Reinforcement Learning Principles.

Reinforcement Learning-Based Control for Nonlinear Discrete-Time Systems with Unknown Control Directions and Control Constraints

Reinforcement-Learning-Based Robust Controller Design for Continuous-Time Uncertain Nonlinear Systems Subject to Input Constraints

Off-Policy Risk-Sensitive Reinforcement Learning-Based Constrained Robust Optimal Control

Reinforcement Learning-Based Control for a Class of Nonlinear Systems with Unknown Control Directions

Model-Based Safe Reinforcement Learning with Time-Varying State and Control Constraints: An Application to Intelligent Vehicles

Data-based Robust Adaptive Control for a Class of Unknown Nonlinear Constrained-Input Systems Via Integral Reinforcement Learning

Adaptive output feedback reinforcement learning control for continuous time switched stochastic nonlinear systems with unknown control coefficients and full-state constraints

Model-Based Safe Reinforcement Learning With Time-Varying Constraints: Applications to Intelligent Vehicles

Robust Near-optimal Control for Constrained Nonlinear System via Integral Reinforcement Learning

Data-Driven Robust Control of Discrete-Time Uncertain Linear Systems Via Off-Policy Reinforcement Learning.

Lagrangian-based online safe reinforcement learning for state-constrained systems

Online Off-Policy Reinforcement Learning for Optimal Control of Unknown Nonlinear Systems Using Neural Networks

Reinforcement Learning for Robust Neuro-Control of Constrained Nonlinear Systems

Robust Reinforcement Learning for Risk-Sensitive Linear Quadratic Gaussian Control

Adaptive Optimized Backstepping Control-Based RL Algorithm for Stochastic Nonlinear Systems with State Constraints and Its Application.

Reinforcement Learning Control of Constrained Dynamic Systems with Uniformly Ultimate Boundedness Stability Guarantee

Integral Reinforcement Learning-Based Guaranteed Cost Control for Unknown Nonlinear Systems Subject to Input Constraints and Uncertainties.

Safe Reinforcement Learning Algorithm and Its Application in Intelligent Control for CPS