Abstract:Remaining competitive in future conflicts with technologically-advanced competitors requires us to accelerate our research and development in artificial intelligence (AI) for wargaming. More importantly, leveraging machine learning for intelligent combat behavior development will be key to one day achieving superhuman performance in this domain--elevating the quality and accelerating the speed of our decisions in future wars. Although deep reinforcement learning (RL) continues to show promising results in intelligent agent behavior development in games, it has yet to perform at or above the human level in the long-horizon, complex tasks typically found in combat modeling and simulation. Capitalizing on the proven potential of RL and recent successes of hierarchical reinforcement learning (HRL), our research is investigating and extending the use of HRL to create intelligent agents capable of performing effectively in these large and complex simulation environments. Our ultimate goal is to develop an agent capable of superhuman performance that could then serve as an AI advisor to military planners and decision-makers. This papers covers our ongoing approach and the first three of our five research areas aimed at managing the exponential growth of computations that have thus far limited the use of AI in combat simulations: (1) developing an HRL training framework and agent architecture for combat units; (2) developing a multi-model framework for agent decision-making; (3) developing dimension-invariant observation abstractions of the state space to manage the exponential growth of computations; (4) developing an intrinsic rewards engine to enable long-term planning; and (5) implementing this framework into a higher-fidelity combat simulation.

Implementing First-Person Shooter Game AI in WILD-SCAV with Rule-Enhanced Deep Reinforcement Learning

Training Interactive Agent in Large FPS Game Map with Rule-enhanced Reinforcement Learning

Playing FPS Games with Environment-Aware Hierarchical Reinforcement Learning

Self-play Reinforcement Learning with Comprehensive Critic in Computer Games

On the Development of an Autonomous Agent for a 3D First-Person Shooter Game Using Deep Reinforcement Learning

Using VizDoom Research Platform Scenarios for Benchmarking Reinforcement Learning Algorithms in First-Person Shooter Games

Playing FPS Games with Deep Reinforcement Learning

Reinforcement Learning Applied to AI Bots in First-Person Shooters: A Systematic Review

Learning to Shoot in First Person Shooter Games by Stabilizing Actions and Clustering Rewards for Reinforcement Learning

Deep Reinforcement Learning for Target Searching in Cognitive Electronic Warfare

Adaptive Shooting for Bots in First Person Shooter Games Using Reinforcement Learning

The Fittest Wins: A Multistage Framework Achieving New SOTA in ViZDoom Competition

Multi-UAV Pursuit-Evasion with Online Planning in Unknown Environments by Deep Reinforcement Learning

Research on Autonomous Manoeuvre Decision Making in Within-Visual-Range Aerial Two-Player Zero-Sum Games Based on Deep Reinforcement Learning

Evaluating reinforcement learning algorithms in first-person shooter games using VizDoom

Deep Reinforcement Learning for Autonomous Ground Vehicle Exploration Without A-Priori Maps

Mastering the Game of 3v3 Snakes with Rule-Enhanced Multi-Agent Reinforcement Learning

Enhancing Offline Coverage Path Planning with Deep Reinforcement Learning

Development of a Simulation Environment for Robot Soccer Game with Deep Reinforcement Learning and Role Assignment

Scaling Intelligent Agents in Combat Simulations for Wargaming

Intelligent Decision-Making and Human Language Communication Based on Deep Reinforcement Learning in a Wargame Environment