Abstract:Guidance commands of flight vehicles are a series of data sets with fixed time intervals, thus guidance design constitutes a sequential decision problem and satisfies the basic conditions for using deep reinforcement learning (DRL). In this paper, we consider the scenario where the escape flight vehicle (EFV) generates guidance commands based on DRL and the pursuit flight vehicle (PFV) generates guidance commands based on the proportional navigation method. For the EFV, the objective of the guidance design entails progressively maximizing the residual velocity, subject to the constraint imposed by the given evasion distance. Thus an irregular dynamic max-min problem of extremely large-scale is formulated, where the time instant when the optimal solution can be attained is uncertain and the optimum solution depends on all the intermediate guidance commands generated before. For solving this problem, a two-step strategy is conceived. In the first step, we use the proximal policy optimization (PPO) algorithm to generate the guidance commands of the EFV. The results obtained by PPO in the global search space are coarse, despite the fact that the reward function, the neural network parameters and the learning rate are designed elaborately. Therefore, in the second step, we propose to invoke the evolution strategy (ES) based algorithm, which uses the result of PPO as the initial value, to further improve the quality of the solution by searching in the local space. Simulation results demonstrate that the proposed guidance design method based on the PPO algorithm is capable of achieving a residual velocity of 67.24 m/s, higher than the residual velocities achieved by the benchmark soft actor-critic and deep deterministic policy gradient algorithms. Furthermore, the proposed ES-enhanced PPO algorithm outperforms the PPO algorithm by 2.7\%, achieving a residual velocity of 69.04 m/s.

Intelligent Diving Guidance with Terminal Angle and Velocity Constraints Via Deep Reinforcement Learning

Multi-constrained Intelligent Gliding Guidance Via Optimal Control and DQN

Model-free Maneuvering Control of Fixed-Wing UAVs Based on Deep Reinforcement Learning

A Multi-Constraint Guidance and Maneuvering Penetration Strategy via Meta Deep Reinforcement Learning

High-dynamic Intelligent Maneuvering Guidance Strategy Via Deep Reinforcement Learning

Integrated diving guidance strategy with multiple constraints

Guidance Design for Escape Flight Vehicle Using Evolution Strategy Enhanced Deep Reinforcement Learning

Terminal Adaptive Guidance for Autonomous Hypersonic Strike Weapons via Reinforcement Learning

Autonomous UAV Navigation with Adaptive Control Based on Deep Reinforcement Learning

Adaptive Guidance and Integrated Navigation with Reinforcement Meta-Learning

Online Trajectory Planning Method for Midcourse Guidance Phase Based on Deep Reinforcement Learning

A Domain-Knowledge-Aided Deep Reinforcement Learning Approach for Flight Control Design

A Vision Based Deep Reinforcement Learning Algorithm for UAV Obstacle Avoidance

A unified intelligent control strategy synthesizing multi-constrained guidance and avoidance penetration

Integrated entry guidance with no-fly zone constraint using reinforcement learning and predictor-corrector technique

Target tracking strategy using deep deterministic policy gradient

Single degree of freedom control based on deep reinforcement learning for underwater unmanned vehicle

Deep Reinforcement Learning for Safe Landing Site Selection with Concurrent Consideration of Divert Maneuvers

Deep Reinforcement Learning With Application to Air Confrontation Intelligent Decision-Making of Manned/Unmanned Aerial Vehicle Cooperative System

Target Tracking Control of a Biomimetic Underwater Vehicle Through Deep Reinforcement Learning

AUV Path Following Control using Deep Reinforcement Learning Under the Influence of Ocean Currents.