Abstract:Online planning is crucial for high performance in many complex sequential decision-making tasks. Monte Carlo Tree Search (MCTS) employs a principled mechanism for trading off exploration for exploitation for efficient online planning, and it outperforms comparison methods in many discrete decision-making domains such as Go, Chess, and Shogi. Subsequently, extensions of MCTS to continuous domains have been developed. However, the inherent high branching factor and the resulting explosion of the search tree size are limiting the existing methods. To address this problem, we propose Continuous Monte Carlo Graph Search (CMCGS), an extension of MCTS to online planning in environments with continuous state and action spaces. CMCGS takes advantage of the insight that, during planning, sharing the same action policy between several states can yield high performance. To implement this idea, at each time step, CMCGS clusters similar states into a limited number of stochastic action bandit nodes, which produce a layered directed graph instead of an MCTS search tree. Experimental evaluation shows that CMCGS outperforms comparable planning methods in several complex continuous DeepMind Control Suite benchmarks and 2D navigation and exploration tasks with limited sample budgets. Furthermore, CMCGS can be scaled up through parallelization, and it outperforms the Cross-Entropy Method (CEM) in continuous control with learned dynamics models.

Adaptive CGF Commander Behavior Modeling Through HTN Guided Monte Carlo Tree Search

Extending Htn To Planning And Execution Control For Small Combat Unit Simulation

Hierarchical Decision and Control for Continuous Multitarget Problem: Policy Evaluation with Action Delay

A Hybrid Approach to Planning and Execution in Dynamic Environments Through Hierarchical Task Networks and Behavior Trees

Application of Hierarchical Task Network Planning to Uninhabited Combat Aerial Vehicles Planning

Chebyshev Metric Based Multi-Objective Monte Carlo Tree Search for Combat Simulations

Self-play Decision-making Method of Deep Reinforcement Learning Guided by Behavior Tree under Complex Environment

Modeling and analysis of command and control system in cooperative air combat

Behavior Modeling Method of CGF in Naval Battle Simulation

Decision-Making Method for Space-Based Combat Based on Hierarchical Structure Varied Discrete Dynamic Bayesian Network

Applying Monte-Carlo Tree Search in HTN Planning

A Collaborative Decision Making Approach for Multi-Unmanned Combat Vehicles Based on the Behaviour Tree

A Hierarchical Game-Theoretic Decision-Making for Cooperative Multi-Agent Systems Under the Presence of Adversarial Agents

Efficient Object Manipulation Planning with Monte Carlo Tree Search

Multi-agent Task Assignment Based on the Situation Field and Game Theory

Continuous Monte Carlo Graph Search

Monte Carlo Tree Search Based Tactical Maneuvering

Monte-Carlo Tree Search for Behavior Planning in Autonomous Driving

Research on Knowledge Graph Completion Model Combining Temporal Convolutional Network and Monte Carlo Tree Search

Hierarchical planning-based crowd formation: HPCF

Decentralized Cooperative Planning for Automated Vehicles with Hierarchical Monte Carlo Tree Search