Abstract:Online model-free reinforcement learning (RL) approaches play a crucial role in coping with the real-world applications, such as the behavioral decision making in robotics. How to balance the exploration and exploitation processes is a central problem in RL. A balanced ratio of exploration/exploitation has a great influence on the total learning time and the quality of the learned strategy. Therefore, various action selection policies have been presented to obtain a balance between the exploration and exploitation procedures. However, these approaches are rarely, automatically, and dynamically regulated to the environment variations. One of the most amazing self-adaptation mechanisms in animals is their capacity to dynamically switch between exploration and exploitation strategies. This article proposes a novel neurophysiologically motivated model which simulates the role of medial prefrontal cortex (MPFC) and lateral prefrontal cortex (LPFC) in behavior decision. The sensory input is transmitted to the MPFC, then the ventral tegmental area (VTA) receives a reward and calculates a dopaminergic reinforcement signal, and the feedback categorization neurons in anterior cingulate cortex (ACC) calculate the vigilance according to the dopaminergic reinforcement signal. Then the vigilance is transformed to LPFC to regulate the exploration rate, finally the exploration rate is transmitted to thalamus to calculate the corresponding action probability. This action selection mechanism is introduced to the actor–critic model of the basal ganglia, combining with the cerebellum model based on the developmental network to construct a new hybrid neuromodulatory model to select the action of the agent. Both the simulation comparison with other four traditional action selection policies and the physical experiment results demonstrate the potential of the proposed neuromodulatory model in action selection.

Action Selection Based on Prediction for Robot Planning

Robust Real-Time Motion Retargeting Via Neural Latent Prediction

Predicting Future Actions of Reinforcement Learning Agents

Future Predictive Success-or-Failure Classification for Long-Horizon Robotic Tasks

Active Predicting Coding: Brain-Inspired Reinforcement Learning for Sparse Reward Robotic Control Problems

Multi-step planning with learned effects of partial action executions

Image-based Action Generation Method using State Prediction and Cost Estimation Learning

Goal-Directed Behavior under Variational Predictive Coding: Dynamic Organization of Visual Attention and Working Memory

Planning Coordinated Human-Robot Motions with Neural Network Full-Body Prediction Models

Self-Supervised Visual Planning with Temporal Skip Connections

Visual Robot Task Planning

Action-Conditional Video Prediction using Deep Networks in Atari Games

A brain-inspired intention prediction model and its applications to humanoid robot

Autoregressive Action Sequence Learning for Robotic Manipulation

Developing Robot Reaching Skill Via Look-ahead Planning

Active Exploration Deep Reinforcement Learning for Continuous Action Space with Forward Prediction

Behavior Decision of Mobile Robot With a Neurophysiologically Motivated Reinforcement Learning Model

Robot Active Neural Sensing and Planning in Unknown Cluttered Environments

Efficient multitask learning with an embodied predictive model for door opening and entry with whole-body control

Adaptive Motion Generation Using Uncertainty-Driven Foresight Prediction

Unsupervised Learning of Effective Actions in Robotics