Abstract:The autonomous decision-making of robots in searching and rescue tasks is of great significance for reducing the risk to human rescuers. To make the robot generate decision-making autonomously and path planning reasonably in the face of complex searching and rescue tasks with multi-solution, an off-policy hierarchical reinforcement learning algorithm was designed in this paper. The algorithm consists of two layers of Soft Actor-Critic（SAC） agents, where the higher-level agent can automatically generate goals needed by the lower-level agent and can provide intrinsic reward to guide the lower-level agent to interact with the environment directly. Under the framework of hierarchical reinforcement learning, the robot searching and rescue task in a complex interactive environment was first described as a two-layer structure with a high-level semi-Markov decision process and a low-level Markov decision process. Then different state spaces, action spaces and reward functions at different levels were designed. Next, in view of the problem that the goals and reward functions in traditional reinforcement learning algorithms were needed to design manually, a SAC-based off-policy hierarchical reinforcement learning algorithm was applied to train bipedal mobile robots to interact with the complex environment. The autonomous decision-making of the searching and rescue robots was achieved through efficient use of data and adjustment of goal space. The simulation results verify the effectiveness and generality of the proposed algorithm in solving complex multi-path searching and rescue tasks.

Automatic hierarchical approach of MAXQ based on action space partition

Automatic Discovery and Transfer of Maxq Hierarchies in A Complex System

Parallel Automatic Hierarchy in Hierarchical Reinforcement Learning

Multi-Agent Hierarchical Reinforcement Learning by Integrating Options into MAXQ

Dynamic Hierarchies in Hierarchical Reinforcement Learning

Model-based learning with Bayesian and MAXQ value function decomposition for hierarchical task

Hierarchical Reinforcement Learning Algorithm Based on Structural State-Space

Automatic formation of the structure of abstract machines in hierarchical reinforcement learning with state clustering

Option Automatic Generation in Hierarchical Reinforcement Learning

New Method of Hierarchical Reinforcement Learning

State Abstraction in MAXQ Hierarchical Reinforcement Learning

Hierarchical Reinforcement Learning with OMQ

Approximate planning for bayesian hierarchical reinforcement learning

Hierarchical Reinforcement Learning with an Automatically Generated Hierarchy Based on Immune Clustering

Algorithm for Automatic Constructing Option Based on Multi-Agent

Hierarchical Method for Cooperative Multiagent Reinforcement Learning in Markov Decision Processes

Adjacency Constraint for Efficient Hierarchical Reinforcement Learning

Algorithms for Batch Hierarchical Reinforcement Learning

A Hierarchical Reinforcement Learning Algorithm Based on Heuristic Reward Function

Hierarchical Reinforcement Learning Based on System Model

Autonomous Decision-making of Searching and Rescue Robots Based on Off-policy Hierarchical Reinforcement Learning in a Complex Interactive Environment