Abstract:The reconnaissance of high-value targets is prerequisite for effective operations. The recent appreciation of deep reinforcement learning (DRL) arises from its success in navigation problems, but due to the competitiveness and complexity of the military field, the applications of DRL in the military field are still unsatisfactory. In this paper, an end-to-end DRL-based intelligent reconnaissance mission planning is proposed for dual unmanned aerial vehicle (dual UAV) cooperative reconnaissance missions under high-threat and dense situations. Comprehensive consideration is given to specific mission properties and parameter requirements through the whole modelling. Firstly, the reconnaissance mission is described as a Markov decision process (MDP), and the mission planning model based on DRL is established. Secondly, the environment and UAV motion parameters are standardized to input the neural network, aiming to deduce the difficulty of algorithm convergence. According to the concrete requirements of non-reconnaissance by radars, dual-UAV cooperation and wandering reconnaissance in the mission, four reward functions with weights are designed to enhance agent understanding to the mission. To avoid sparse reward, the clip function is used to control the reward value range. Finally, considering the continuous action space of reconnaissance mission planning, the widely applicable proximal policy optimization (PPO) algorithm is used in this paper. The simulation is carried out by combining offline training and online planning. By changing the location and number of ground detection areas, from 1 to 4, the model with PPO can maintain 20% of reconnaissance proportion and a 90% mission complete rate and help the reconnaissance UAV to complete efficient path planning. It can adapt to unknown continuous high-dimensional environmental changes, is generalizable, and reflects strong intelligent planning performance.

An intelligent generating method for multi-target attacking strategy based on environment-aware deep reinforcement learning

Cooperative multi-agent target searching: a deep reinforcement learning approach based on parallel hindsight experience replay

Adaptive Deep Reinforcement Learning for Non-Stationary Environments

Autonomous Decision Making for UAV Cooperative Pursuit-Evasion Game with Reinforcement Learning

Multi-intent autonomous decision-making for air combat with deep reinforcement learning

Multi-UAV simultaneous target assignment and path planning based on deep reinforcement learning in dynamic multiple obstacles environments

Adversarial Decision-Making for Moving Target Defense: A Multi-Agent Markov Game and Reinforcement Learning Approach

Deep Reinforcement Learning for Target Searching in Cognitive Electronic Warfare

Deep Reinforcement Learning With Application to Air Confrontation Intelligent Decision-Making of Manned/Unmanned Aerial Vehicle Cooperative System

Self-play Decision-making Method of Deep Reinforcement Learning Guided by Behavior Tree under Complex Environment

Deep Reinforcement-Learning-Based Air-Combat-Maneuver Generation Framework

Deep Reinforcement Learning‐Based Air Defense Decision‐Making Using Potential Games

Research and Implementation of Intelligent Decision Based on a Priori Knowledge and DQN Algorithms in Wargame Environment

Adaptive trajectory-constrained exploration strategy for deep reinforcement learning

Learning Multi-Pursuit Evasion for Safe Targeted Navigation of Drones

Deep Reinforcement Learning for Intelligent Dual-UAV Reconnaissance Mission Planning

Multi-Agent Guided Deep Reinforcement Learning Approach Against State Perturbed Adversarial Attacks

Improving anti-jamming decision-making strategies for cognitive radar via multi-agent deep reinforcement learning

Research on Autonomous Manoeuvre Decision Making in Within-Visual-Range Aerial Two-Player Zero-Sum Games Based on Deep Reinforcement Learning

Generating intelligent agent behaviors in multi-agent game AI using deep reinforcement learning algorithm

Towards Real-Time Path Planning through Deep Reinforcement Learning for a UAV in Dynamic Environments