Abstract:Online model-free reinforcement learning (RL) approaches play a crucial role in coping with the real-world applications, such as the behavioral decision making in robotics. How to balance the exploration and exploitation processes is a central problem in RL. A balanced ratio of exploration/exploitation has a great influence on the total learning time and the quality of the learned strategy. Therefore, various action selection policies have been presented to obtain a balance between the exploration and exploitation procedures. However, these approaches are rarely, automatically, and dynamically regulated to the environment variations. One of the most amazing self-adaptation mechanisms in animals is their capacity to dynamically switch between exploration and exploitation strategies. This article proposes a novel neurophysiologically motivated model which simulates the role of medial prefrontal cortex (MPFC) and lateral prefrontal cortex (LPFC) in behavior decision. The sensory input is transmitted to the MPFC, then the ventral tegmental area (VTA) receives a reward and calculates a dopaminergic reinforcement signal, and the feedback categorization neurons in anterior cingulate cortex (ACC) calculate the vigilance according to the dopaminergic reinforcement signal. Then the vigilance is transformed to LPFC to regulate the exploration rate, finally the exploration rate is transmitted to thalamus to calculate the corresponding action probability. This action selection mechanism is introduced to the actor–critic model of the basal ganglia, combining with the cerebellum model based on the developmental network to construct a new hybrid neuromodulatory model to select the action of the agent. Both the simulation comparison with other four traditional action selection policies and the physical experiment results demonstrate the potential of the proposed neuromodulatory model in action selection.

Connecting Model-Based and Model-Free Control With Emotion Modulation in Learning Systems

Computational Modeling of Emotion-Motivated Decisions for Continuous Control of Mobile Robots

An Emotion-Based Approach to Decision Making and Self Learning in Autonomous Robot Control

Model-Based Robot Learning Control with Uncertainty Directed Exploration

A Emotion Development Agent Model Based On Occ Model And Operant Conditioning

FCS-MPC of Power Converters: an Event-Driven Brain Emotional Learning Approach

Learning-Based Neural Dynamic Surface Predictive Control for MMC

A Brain-Inspired Decision-Making Method for Upper Limb Exoskeleton Based on Multi-Brain-Region Structure and Multimodal Information Fusion

Emotion Regulation Based on Multi-objective Weighted Reinforcement Learning for Human-robot Interaction

An Artificial Emotion model Baesd-on State-space Method

Implementation of an Artificial Emotion Model and Behavior Strategies Based on Finite Automata

Research on Emotion Theory and the Decision Models Based on Emotion

Artificial Emotion Model Based on Neuromodulators and Q-learning

Behavior Decision of Mobile Robot With a Neurophysiologically Motivated Reinforcement Learning Model

A Computational Model of Emotion Based on Audio-Visual Stimuli Understanding and Personalized Regulation with Concurrency

Self-Organizing Double Function-Link Fuzzy Brain Emotional Control System Design for Uncertain Nonlinear Systems.

Model-Free Robust Optimal Feedback Mechanisms of Biological Motor Control

A Novel Self-Organizing Emotional CMAC Network for Robotic Control

Model-Based Control with Sparse Neural Dynamics

Emotion as an emergent phenomenon of the neurocomputational energy regulation mechanism of a cognitive agent in a decision-making task