Safe Reinforcement Learning Based Balance Control for Multi-Cylinder Hydraulic Press

Chao Jia,Zijian Song,Lifeng Du,Hongkun Wang,Jia Chao
DOI: https://doi.org/10.1115/1.4064992
2024-03-05
Abstract:Abstract Due to load uncertainty and unmodeled dynamics in multi-cylinder hydraulic systems, this paper proposes a balance control algorithm based on safe reinforcement learning to release the restrictions of classical model based control methods that depend on fixed gain. In this paper, the hydraulic press is controlled by a trained agent which directly maps the system states to control commands in an end-to-end manner. By introducing an action modifier into the algorithm, the system states are kept within security constraints from the beginning of training, making safe exploration possible. Furthermore, a normalized exponential reward function has been proposed. Compared with a quadratic reward function, the precision is greatly improved under the same training steps. The experiment shows that our algorithm can achieve high precision and fast balance for multi-cylinder hydraulic presses while being highly robust. To the best of our knowledge, this research is the first to attempt the application of a reinforcement learning algorithm to multi execution units of hydraulic systems.
automation & control systems,instruments & instrumentation
What problem does this paper attempt to address?