Abstract:Guidance commands of flight vehicles are a series of data sets with fixed time intervals, thus guidance design constitutes a sequential decision problem and satisfies the basic conditions for using deep reinforcement learning (DRL). In this paper, we consider the scenario where the escape flight vehicle (EFV) generates guidance commands based on DRL and the pursuit flight vehicle (PFV) generates guidance commands based on the proportional navigation method. For the EFV, the objective of the guidance design entails progressively maximizing the residual velocity, subject to the constraint imposed by the given evasion distance. Thus an irregular dynamic max-min problem of extremely large-scale is formulated, where the time instant when the optimal solution can be attained is uncertain and the optimum solution depends on all the intermediate guidance commands generated before. For solving this problem, a two-step strategy is conceived. In the first step, we use the proximal policy optimization (PPO) algorithm to generate the guidance commands of the EFV. The results obtained by PPO in the global search space are coarse, despite the fact that the reward function, the neural network parameters and the learning rate are designed elaborately. Therefore, in the second step, we propose to invoke the evolution strategy (ES) based algorithm, which uses the result of PPO as the initial value, to further improve the quality of the solution by searching in the local space. Simulation results demonstrate that the proposed guidance design method based on the PPO algorithm is capable of achieving a residual velocity of 67.24 m/s, higher than the residual velocities achieved by the benchmark soft actor-critic and deep deterministic policy gradient algorithms. Furthermore, the proposed ES-enhanced PPO algorithm outperforms the PPO algorithm by 2.7\%, achieving a residual velocity of 69.04 m/s.

Enhancing the landing guidance of a reusable launch vehicle by improving genetic algorithm-based deep reinforcement learning using Hybrid Deterministic-Stochastic algorithm

Path Following for Autonomous Ground Vehicle Using DDPG Algorithm: A Reinforcement Learning Approach

Deep Reinforcement Learning Automatic Landing Control of Fixed-Wing Aircraft Using Deep Deterministic Policy Gradient

Automatic Landing Control of Aircraft Based on Cognitive Load Theory and DDPG

Precision Landing of Autonomous Parafoil System Via Deep Reinforcement Learning

Deep Reinforcement Learning using Genetic Algorithm for Parameter Optimization

Homing Guidance Law Design against Maneuvering Targets Based on DDPG

Multi-point powered descent guidance based on optimal sensitivity

Lander.AI: Adaptive Landing Behavior Agent for Expertise in 3D Dynamic Platform Landings

Model Predictive Guidance for Fuel-Optimal Landing of Reusable Launch Vehicles

GARL: Genetic Algorithm-Augmented Reinforcement Learning to Detect Violations in Marker-Based Autonomous Landing Systems

Optimal trajectory-tracking guidance for reusable launch vehicle based on adaptive dynamic programming

Optimal Control Homing of Rocket First-Stage Booster Recovery Based on Improved Genetic Algorithm with SQP

A Deep Reinforcement Learning Strategy for UAV Autonomous Landing on a Platform

Multi-constrained Intelligent Gliding Guidance Via Optimal Control and DQN

Optimization of UAV Robust Control Using Genetic Algorithm

Coupling of Advanced Guidance and Robust Control for the Descent and Precise Landing of Reusable Launchers

Guidance Design for Escape Flight Vehicle Using Evolution Strategy Enhanced Deep Reinforcement Learning

Calculate the ignition height of the vertical landing phase online for the reusable rocket

Real-time Control for Fuel-Optimal Moon Landing Based on an Interactive Deep Reinforcement Learning Algorithm

A decentralized path planning model based on deep reinforcement learning