Abstract:Wargames are essential simulators for various war scenarios. However, the increasing pace of warfare has rendered traditional wargame decision-making methods inadequate. To address this challenge, wargame-assisted decision-making methods that leverage artificial intelligence techniques, notably reinforcement learning, have emerged as a promising solution. The current wargame environment is beset by a large decision space and sparse rewards, presenting obstacles to optimizing decision-making methods. To overcome these hurdles, a Multi-Agent Deep Deterministic Policy Gradient (MADDPG) based wargame decision-making method is presented. The Partially Observable Markov Decision Process (POMDP), joint action-value function, and the Gumbel-Softmax estimator are applied to optimize MADDPG in order to adapt to the wargame environment. Furthermore, a wargame decision-making method based on the improved MADDPG algorithm is proposed. Using supervised learning in the proposed approach, the training efficiency is improved and the space for manipulation before the reinforcement learning phase is reduced. In addition, a policy gradient estimator is incorporated to reduce the action space and to obtain the global optimal solution. Furthermore, an additional reward function is designed to address the sparse reward problem. The experimental results demonstrate that our proposed wargame decision-making method outperforms the pre-optimization algorithm and other algorithms based on the AC framework in the wargame environment. Our approach offers a promising solution to the challenging problem of decision-making in wargame scenarios, particularly given the increasing speed and complexity of modern warfare.

Battlefield Agent Decision-Making Based on Markov Decision Process.

Decision-Making Method for Space-Based Combat Based on Hierarchical Structure Varied Discrete Dynamic Bayesian Network

Research on Wargame Decision-Making Method Based on Multi-Agent Deep Deterministic Policy Gradient

Dynamic Decision-Making Modeling for Army Operation Command Post Entity

Modeling of Military Decision-Making Based on Multi-Agent Influence Diagrams and Games

Dynamic Decision Making Method for Cooperation of Airborne Hardkill /softkill Weapons

Solving nonstationary Markov decision processes via contextual decomposition: A military air battle management application

Robust Tactical Decision Making for Deep Uncertainty Battlefield Environment

Adversarial Decision-Making for Moving Target Defense: A Multi-Agent Markov Game and Reinforcement Learning Approach

Decision-making of Emergent Incident based on P-MADDPG

Multi-attribute Decision-Making in Wargames Leveraging the Entropy-Weight Method in Conjunction with Deep Reinforcement Learning

Mission Decision-Making for UAV under Dynamic Environment

Multi-Agent Air Combat Decision-making Based on Battlefield Attention Information

Self-play Decision-making Method of Deep Reinforcement Learning Guided by Behavior Tree under Complex Environment

Decision-making Model of Tactics Maneuver Agent Based on Fuzzy Markov Decision Theory

Markov Decision Process For Automatic Cyber Defense

Multiple Attribute Decision Making Approach to Attacking Order of Battlefields Targets

Study on air formation to ground attack-defends decision making in fuzzy condition

RPD Decision-making Model Based on Composite Agent

A Collaborative Combat Decision-Making Method Based on Multi-Agent Deep Reinforcement Learning

A Neural Network-Based Intelligent Decision-Making in the Air-Offensive Campaign with Simulation