Autonomous Shape Decision Making of Morphing Aircraft with Improved Reinforcement Learning

Weilai Jiang,Chenghong Zheng,Delong Hou,Kangsheng Wu,Yaonan Wang

DOI: https://doi.org/10.3390/aerospace11010074

IF: 2.66

2024-01-13

Aerospace

Abstract:The autonomous shape decision-making problem of a morphing aircraft (MA) with a variable wingspan and sweep angle is studied in this paper. Considering the continuity of state space and action space, a more practical autonomous decision-making algorithm framework of MA is designed based on the deep deterministic policy gradient (DDPG) algorithm. Furthermore, the DDPG with a task classifier (DDPGwTC) algorithm is proposed in combination with the long short-term memory (LSTM) network to improve the convergence speed of the algorithm. The simulation results show that the shape decision-making algorithm based on the DDPGwTC enables MA to adopt the optimal morphing strategy in different task environments with higher autonomy and environmental adaptability, which verifies the effectiveness of the proposed algorithm.

engineering, aerospace

What problem does this paper attempt to address?

### Problems Addressed by the Paper This paper primarily investigates the autonomous morphing decision-making problem for Morphing Aircraft (MA). Specifically, it focuses on morphing aircraft with variable wingspan and sweep angle, and proposes an improved reinforcement learning algorithm to address this issue. #### Main Work and Contributions: 1. **Algorithm Framework Design**: Considering the continuity of the aircraft's state space and action space, the paper adopts the Deep Deterministic Policy Gradient (DDPG) algorithm to design a more practical autonomous decision-making algorithm framework. 2. **Introduction of Task Classifier**: To improve the convergence speed of the algorithm, the paper combines the Long Short-Term Memory (LSTM) network to design a task classifier, proposing the DDPG algorithm with Task Classifier (DDPGwTC). Through the task classifier, the algorithm can better adapt to different task environments, thereby improving the convergence speed of the algorithm. 3. **Simulation Verification**: Simulation results show that the morphing decision algorithm based on DDPGwTC enables the morphing aircraft to adopt the optimal morphing strategy in different task environments, verifying the effectiveness of the proposed algorithm. ### Brief Summary This paper proposes a method based on an improved reinforcement learning algorithm (DDPGwTC) to address the autonomous morphing decision-making problem for morphing aircraft in complex flight environments. By introducing a task classifier and LSTM network, the algorithm's convergence speed and environmental adaptability are improved, and the effectiveness of the method is ultimately verified through simulation experiments.

Autonomous Shape Decision Making of Morphing Aircraft with Improved Reinforcement Learning

Model-free Maneuvering Control of Fixed-Wing UAVs Based on Deep Reinforcement Learning

Improved Adaptive-Reinforcement Learning Control for morphing unmanned air vehicles

Deep Learning Based Adaptive Deformation of Aerodynamic Shape for Ducted Propellers

Deep Reinforcement Learning With Application to Air Confrontation Intelligent Decision-Making of Manned/Unmanned Aerial Vehicle Cooperative System

cc-DRL: a Convex Combined Deep Reinforcement Learning Flight Control Design for a Morphing Quadrotor

Reinforcement Learning-Based Robust Tracking Control Application to Morphing Aircraft.

Autonomous maneuver decision-making for a UCAV in short-range aerial combat based on an MS-DDQN algorithm

Autonomous flight performance optimization of fixed-wing unmanned aerial vehicle with morphing wingtip

An Intelligent Maneuver Decision-Making Approach for Air Combat Based on Deep Reinforcement Learning and Transformer Networks

Multi-intent autonomous decision-making for air combat with deep reinforcement learning

Multi-objective reward shaping for global and local trajectory planning of wing-in-ground crafts based on deep reinforcement learning

Hierarchical Reinforcement Learning from Competitive Self-play for Dual-aircraft formation air combat

Adaptive Neural Control Based on High Order Integral Chained Differentiator for Morphing Aircraft

Multidisciplinary Performance Enhancement on a Fixed-wing Unmanned Aerial Vehicle via Simultaneous Morphing Wing and Control System Design

A novel inverse design method for morphing airfoil based on deep reinforcement learning

Efficient Modeling of Morphing Wing Flight Using Neural Networks and Cubature Rules

Improving Autonomous Separation Assurance through Distributed Reinforcement Learning with Attention Networks

Enhancing multi-UAV air combat decision making via hierarchical reinforcement learning

A review of modelling and analysis of morphing wings

Deep reinforcement learning and its application in autonomous fitting optimization for attack areas of UCAVs