Abstract:Online model-free reinforcement learning (RL) approaches play a crucial role in coping with the real-world applications, such as the behavioral decision making in robotics. How to balance the exploration and exploitation processes is a central problem in RL. A balanced ratio of exploration/exploitation has a great influence on the total learning time and the quality of the learned strategy. Therefore, various action selection policies have been presented to obtain a balance between the exploration and exploitation procedures. However, these approaches are rarely, automatically, and dynamically regulated to the environment variations. One of the most amazing self-adaptation mechanisms in animals is their capacity to dynamically switch between exploration and exploitation strategies. This article proposes a novel neurophysiologically motivated model which simulates the role of medial prefrontal cortex (MPFC) and lateral prefrontal cortex (LPFC) in behavior decision. The sensory input is transmitted to the MPFC, then the ventral tegmental area (VTA) receives a reward and calculates a dopaminergic reinforcement signal, and the feedback categorization neurons in anterior cingulate cortex (ACC) calculate the vigilance according to the dopaminergic reinforcement signal. Then the vigilance is transformed to LPFC to regulate the exploration rate, finally the exploration rate is transmitted to thalamus to calculate the corresponding action probability. This action selection mechanism is introduced to the actor–critic model of the basal ganglia, combining with the cerebellum model based on the developmental network to construct a new hybrid neuromodulatory model to select the action of the agent. Both the simulation comparison with other four traditional action selection policies and the physical experiment results demonstrate the potential of the proposed neuromodulatory model in action selection.

A Double-layer Decision-making Model Based on Behavior

An Intelligent Decision-Making Subsystem For Autonomous Soccer Robot

A rule-based decision subsystem design approach for intelligent robot

Analysis and design of decision-making subsystem of soccer robot

Towards a cooperative soccer team in RoboCup

Hierarchical Decision and Control for Continuous Multitarget Problem: Policy Evaluation with Action Delay

A Novel Approach to Increase Control Performance of Soccer Robot

Behavior-based implicit planning method and its application to robot soccer system

Multi-Stage Decision-Making Skill Learning for Soccer Robot

Study and Application of Reinforcement Learning in Cooperative Strategy of the Robot Soccer Based on BDI Model

Modeling and Control Architecture for the Competitive Networked Robot System Based on POMDP

Design of Strategy Based on Home and Opponent Information in Robot Soccer Competition

Designing of autonomous humanoid robot soccer system

Cooperation Strategy for Robot Soccer Based on Defuzzification.

Dynamic Formation Planning and Control for Robot Soccer Game with Multi-Agent Reinforcement Learning and Behavioral Model

A New Fuzzy Decision Method for Solving Soccer Robots' Motion Deadlock

Behavior Decision of Mobile Robot With a Neurophysiologically Motivated Reinforcement Learning Model

Research of Double Token Based Collaborative Strategy for Multi-robot System

An Emotion-Based Approach to Decision Making and Self Learning in Autonomous Robot Control

An interpretable decision-making model for autonomous driving

Integrated model of cerebellal supervised learning and basal ganglia's reinforcement learning for mobile robot behavioral decision-making