Research on Solving Nonlinear Problem of Ball and Beam System by Introducing Detail-Reward Function
Shixuan Yao,Xiaochen Liu,Yinghui Zhang,Ze Cui
DOI: https://doi.org/10.3390/sym14091883
2022-09-09
Symmetry
Abstract:As a complex nonlinear system, the first-order incremental relationship between the state variables of the beam and ball system (BABS) is asymmetric in the definition domain of the variables, and the characteristics of the system do not satisfy the superposition theorem. Studying the balance control of the BABS can help to better grasp the relevant characteristics of the nonlinear system. In this paper, the deep reinforcement learning method is used to study the BABS based on a visual sensor. First, the detail-reward function is designed by observing the control details of the system, and the rationality of the function is proved based on Q-function; secondly, considering and comparing the applicability of image processing methods in ball coordinate location, an intelligent location algorithm is proposed, and the location effects between the algorithms are compared and analyzed; then, combining the nonlinear theory and LQR theory, a reinforcement learning policy model is proposed to linearize near the equilibrium point, which significantly improves the control effect. Finally, experiments are designed to verify the effectiveness of the above methods in the control system. The experimental results show that the design scheme can be effectively applied to the control system of the BABS. It is verified that the introduction of detail-reward mechanism into a deep reinforcement learning algorithm can significantly reduce the complexity of the nonlinear control system and iterative algorithm, and effectively solve nonlinear control problems.