Robust Adaptive Control for a Small Unmanned Helicopter Using Reinforcement Learning

Bin Xian,Xu Zhang,Haonan Zhang,Xun Gu
DOI: https://doi.org/10.1109/tnnls.2021.3085767
IF: 14.255
2021-01-01
IEEE Transactions on Neural Networks and Learning Systems
Abstract:This article presents a novel adaptive controller for a small-size unmanned helicopter using the reinforcement learning (RL) control methodology. The helicopter is subject to system uncertainties and unknown external disturbances. The dynamic unmodeling uncertainties of the system are estimated online by the actor network, and the tracking performance function is optimized via the critic network. The estimation error of the actor-critic network and the external unknown disturbances are compensated via the nonlinear robust component based on the sliding mode control method. The stability of the closed-loop system and the asymptotic convergence of the attitude tracking error are proved via the Lyapunov-based stability analysis. Finally, real-time experiments are performed on a helicopter control testbed. The experimental results show that the proposed controller achieves good control performance.
computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, hardware & architecture
What problem does this paper attempt to address?
This paper attempts to solve the control problem of small unmanned helicopters in the presence of system uncertainties and unknown external disturbances. Specifically, the author proposes an adaptive controller based on Reinforcement Learning (RL) to improve the robustness and adaptability of unmanned helicopters. The controller aims to estimate the unmodeled dynamics uncertainties of the system and compensates for these uncertainties and unknown external disturbances by combining the sliding - mode control method, thereby optimizing the tracking performance. The paper also proves the stability of the closed - loop system and the asymptotic convergence of the attitude - tracking error, and verifies the effectiveness of the proposed controller through real - time experiments. The main contributions of the paper include: 1. Designing an RL - based approximator for online estimation of model uncertainties, which improves the robustness to model uncertainties. 2. Combining a sliding - mode - based robust component to compensate for the approximation error of the RL network and unknown disturbances. 3. Adopting Lyapunov - based stability analysis to prove the stability of the closed - loop system under the combined action of the RL network and the robust control component. 4. Proving the asymptotic convergence of the attitude - tracking error of unmanned helicopters. Through these methods, the paper proposes a control strategy that can achieve the asymptotic convergence of the attitude - tracking error of unmanned helicopters with very little knowledge of the dynamic model and in the presence of unknown external disturbances.