Abstract:The cerebral cortex, cerebellum, and basal ganglia play a central role in flexible learning in mammals. However, how these three structures work together is not fully understood. Recently, it has been suggested that reinforcement learning may be implemented not only in the basal ganglia but also in the cerebellum, as the activity of cerebellar climbing fibers represents reward prediction error. If the same learning mechanism via reward prediction error occurs simultaneously in the basal ganglia and cerebellum, it remains unclear how these two regions co-function. Here, we recorded neuronal activity in the output of cerebellum and basal ganglia, the cerebellar nuclei and substantia nigra pars reticulata, respectively, from ChR2 transgenic rats with high-density Neuropixels probes while optogenetically stimulating the cerebral cortex point-by-point. The temporal response patterns could be categorized into two classes in both cerebellar nuclei and substantia nigra pars reticulata. Among them, the fast excitatory response of the cerebellar nuclei due to the input of mossy fibers and the inhibitory response of the substantia nigra pars reticulata via the direct pathway were synchronized. This coincidence, reproduced in a spiking network simulation based on connectome data, was expected to synchronously activate the cerebral cortex via the thalamus. To further investigate the significance of this synchronous positive feedback, we constructed a reservoir model that mimics the time course of the activity dynamics of cerebral cortex and temporal responses of cerebellar nuclei and substantia nigra pars reticulata. Plasticity of both parallel fiber inputs to Purkinje cell and corticostriatal synapses onto the striatal neurons of the direct pathway was essential for successful learning of a reinforcement learning task. Notably, learning was inhibited when the timing of the cerebellar or basal ganglia output was delayed from the real data by 10 ms; the larger this delay, the slower the learning rate. This necessary temporal precision was observed only when the cerebral cortex operated in the β-to-γ frequency range. These results indicate that coordinated output of the cerebellum and basal ganglia, with input from the cerebral cortex in a narrow frequency band, facilitates brain-wide synergistic reinforcement learning. Thus, our findings contribute to a holistic understanding of the interactions among the cerebellum, basal ganglia, and cerebral cortex.

Modeling flexible behaviour by the interactions between hippocampus and cortex

Rapid context inference in a thalamocortical model using recurrent neural networks

A Neural Circuit Model of Flexible Sensorimotor Mapping: Learning and Forgetting on Multiple Timescales

A cortical circuit mechanism for structural knowledge-based flexible sensorimotor decision-making

A model for integrating elementary neural functions into delayed-response behavior

Biologically plausible models of cognitive flexibility: merging recurrent neural networks with full-brain dynamics

A model of cortical cognitive function using hierarchical interactions of gating matrices in internal agents coding relational representations

Context-invariant beliefs are supported by dynamic reconfiguration of single unit functional connectivity in prefrontal cortex of male macaques

Latent representations in hippocampal network model co-evolve with behavioral exploration of task structure

Fronto-striatal Organization: Defining Functional and Microstructural Substrates of Behavioural Flexibility

Synergistic reinforcement learning by cooperation of the cerebellum and basal ganglia

A theory of cerebral learning regulated by the reward system. I. Hypotheses and mathematical description

Building and Breaking the Chain: A Model of Reward Prediction Error Integration and Segmentation of Memory

Discovering Latent States for Model Learning: Applying Sensorimotor Contingencies Theory and Predictive Processing to Model Context

Neurodynamics of the Prefrontal Cortex during Conditional Visuomotor Associations

Context-dependent selection of visuomotor maps

A Neural Model of the Frontal Eye Fields with Reward-Based Learning.

Learning, memory and consolidation mechanisms for behavioral control in hierarchically organized cortico‐basal ganglia systems

Individual variability of neural computations underlying flexible decisions

Partially dissociable roles of the Orbitofrontal cortex and dorsal Hippocampus in context-dependent (hierarchical) reward predictions and contextual inference in learning

Biomimetic model of corticostriatal micro-assemblies discovers new neural code