Abstract:In a complex and uncertain world, how do we select appropriate behavior? One possibility is that we choose actions that are highly reinforced by their probabilistic consequences (model-free processing). However, we may instead plan actions prior to their actual execution by predicting their consequences (model-based processing). It has been suggested that the brain contains multiple yet distinct systems involved in reward prediction. Several studies have tried to allocate model-free and model-based systems to the striatum and the lateral prefrontal cortex (LPFC), respectively. Although there is much support for this hypothesis, recent research has revealed discrepancies. To understand the nature of the reward prediction systems in the LPFC and the striatum, a series of single-unit recording experiments were conducted. LPFC neurons were found to infer the reward associated with the stimuli even when the monkeys had not yet learned the stimulus-reward (SR) associations directly. Striatal neurons seemed to predict the reward for each stimulus only after directly experiencing the SR contingency. However, the one exception was “Exclusive Or” situations in which striatal neurons could predict the reward without direct experience. Previous single-unit studies in monkeys have reported that neurons in the LPFC encode category information, and represent reward information specific to a group of stimuli. Here, as an extension of these, we review recent evidence that a group of LPFC neurons can predict reward specific to a category of visual stimuli defined by relevant behavioral responses. We suggest that the functional difference in reward prediction between the LPFC and the striatum is that while LPFC neurons can utilize abstract code, striatal neurons can code individual associations between stimuli and reward but cannot utilize abstract code.

Distributional reinforcement learning in prefrontal cortex

Anterior cingulate learns reward distribution

An opponent striatal circuit for distributional reinforcement learning

Frontal Cortex and Reward-Guided Learning and Decision-Making

Prefrontal cortex as a meta-reinforcement learning system

A distributional code for value in dopamine-based reinforcement learning

Mechanisms of Hierarchical Reinforcement Learning in Corticostriatal Circuits 1: Computational Analysis

Dissociable Functions of Reward Inference in the Lateral Prefrontal Cortex and the Striatum

Ventrolateral prefrontal cortex in macaques guides decisions in different learning contexts

Heterogeneous reward signals in prefrontal cortex

Ventral frontostriatal circuitry mediates the computation of reinforcement from symbolic gains and losses

Reward Prediction in Prefrontal Cortex and Striatum

Believing in dopamine

Foundations of Multivariate Distributional Reinforcement Learning

An introduction to reinforcement learning for neuroscience

Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective

Dopamine Release Plateau and Outcome Signals in Dorsal Striatum Contrast with Classic Reinforcement Learning Formulations

From conflict management to reward-based decision making: actors and critics in primate medial frontal cortex

Neural interactions in the human frontal cortex dissociate reward and punishment learning

Behavioral and Neural Properties of Social Reinforcement Learning.

Reward Prediction Based on Stimulus Categorization in Primate Lateral Prefrontal Cortex