Abstract:In psychology, goal-setting theory, which has been studied by psychologists for over 35 years, reveals that goals play significant roles in incentive, action and performance for human beings. Based on this theory, a goal net model has been proposed to design intelligent agents that can be viewed as a soft copy of human being somehow. The goal net model has been successfully applied in many agents, specially, non-player-character agents in computer games. Such an agent selects the optimal solution in all possible solutions found by using a recursive algorithm. However, if a goal net is very complex, the time of selection could be too long for the agent to respond quickly when the agent needs to re-select a new solution against the world’s change. Moreover, in some dynamic environments, it is impossible to know the exact outcome of choosing a solution in advance, and so the possible solutions cannot be evaluated precisely. Thus, to address the problem, this paper applies learning algorithm into goal selection in dynamic environments. More specifically, we first develop a reorganization algorithm that can convert a goal net to its equivalent counterpart that a Q-learning algorithm can operate on; then, we define the key component of Q-learning, reward function, according to the feature of goal nets; and finally lots of experiments are conducted to show that, in dynamic environments, the agent with the learning algorithm significantly outperforms the one with the recursive searching algorithm. Therefore, our work suggests an agent model that can effectively be applied in dynamic time-sensitive domain, like computer games and the P2P systems of online movie watching.

A Goal-Conditioned Reinforcement Learning Algorithm with Environment Modeling

Learning Hierarchical Graph-Based Policy for Goal-Reaching in Unknown Environments

Adaptive Environment Modeling Based Reinforcement Learning for Collision Avoidance in Complex Scenes

Goal-Conditioned Reinforcement Learning: Problems and Solutions

Autonomous navigation of mobile robots in unknown environments using off-policy reinforcement learning with curriculum learning

Navigation in Unknown Dynamic Environments Based on Deep Reinforcement Learning

Multigoal Visual Navigation With Collision Avoidance via Deep Reinforcement Learning

Learning World Models for Unconstrained Goal Navigation

Autonomous Learning and Navigation of Mobile Robots Based on Deep Reinforcement Learning

Self-imitation guided goal-conditioned reinforcement learning

Hierarchical reinforcement learning for handling sparse rewards in multi-goal navigation

Goal-Conditioned Reinforcement Learning with Disentanglement-based Reachability Planning

Visual Navigation with Multiple Goals Based on Deep Reinforcement Learning

Improving Generalization in Reinforcement Learning Training Regimes for Social Robot Navigation

Adaptive goal selection for agents in dynamic environments

TLDR: Unsupervised Goal-Conditioned RL via Temporal Distance-Aware Representations

Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigation

Learning Complicated Navigation Skills from Limited Experience Via Augmenting Offline Datasets

Goal Exploration via Adaptive Skill Distribution for Goal-Conditioned Reinforcement Learning

GOPlan: Goal-conditioned Offline Reinforcement Learning by Planning with Learned Models

Deep Reinforcement Learning with Heuristic Corrections for UGV Navigation