Reinforcement Learning for Dual-Control Aircraft Six-Degree-of-Freedom Attitude Control with System Uncertainty

Yuqi Yuan,Di Zhou
DOI: https://doi.org/10.3390/aerospace11040281
IF: 2.66
2024-04-05
Aerospace
Abstract:This article proposes a near-optimal control strategy based on reinforcement learning, which is applied to the six-degree-of-freedom (6-DoF) attitude control of dual-control aircraft. In order to solve the problem that the existing reinforcement learning is difficult to apply to the high-dimensional multiple-input multiple-output (MIMO) systems, the Long Short-Term Memory (LSTM) neural network is introduced to replace the polynomial network in the adaptive dynamic programming (ADP) technique. Meanwhile, based on the Lyapunov method, a novel online adaptive updating law of LSTM neural network weights is given, and the stability of the system is verified. In the simulation process, the algorithm proposed in this article is applied to the six-degree-of-freedom attitude control problem of dual-control aircraft with system uncertainty. The simulation results show that the algorithm can achieve near-optimal control.
engineering, aerospace
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how to perform six - degree - of - freedom (6 - DoF) attitude control of dual - control aircraft in the presence of system uncertainties. Specifically, the paper proposes an approximately optimal control strategy based on reinforcement learning to address the control challenges of high - dimensional multi - input multi - output (MIMO) systems. By introducing the long - short - term memory (LSTM) neural network to replace the polynomial network in the adaptive dynamic programming (ADP) technique and designing an online adaptive update law for the LSTM neural network weights based on the Lyapunov method, the paper verifies the stability of the system. The main contributions of the paper include: 1. Proposing a reinforcement - learning - based approximately optimal control method using the LSTM neural network, which is applied to the 6 - DoF attitude control of dual - control aircraft. Unlike existing algorithms, this algorithm does not need to decouple the nonlinear aircraft attitude dynamics model, retains the internal characteristics of the system as much as possible, and effectively solves the optimal control problem of MIMO nonlinear control systems. 2. Based on the nonlinear optimal control theory, introducing an additional term based on output feedback to ensure that the closed - loop system with perturbations is bounded and converges within a small neighborhood of the control command. 3. Based on the Lyapunov method, presenting an online adaptive update law for the LSTM neural network weights. All update laws are in analytical form, avoiding the system burden caused by large - scale real - time operations, and proving the stability of the system. 4. Verifying through simulation analysis that the algorithm can effectively solve the 6 - DoF attitude control problem of dual - control aircraft. The specific content of the paper includes establishing the attitude dynamics model of dual - control aircraft, designing the optimal control law based on the HJB equation, designing the approximately optimal control law based on the LSTM neural network and output feedback, and designing the online update law for the LSTM neural network weights. Finally, the effectiveness of the algorithm is verified through simulation analysis.