Abstract:The cerebral cortex, cerebellum, and basal ganglia play a central role in flexible learning in mammals. However, how these three structures work together is not fully understood. Recently, it has been suggested that reinforcement learning may be implemented not only in the basal ganglia but also in the cerebellum, as the activity of cerebellar climbing fibers represents reward prediction error. If the same learning mechanism via reward prediction error occurs simultaneously in the basal ganglia and cerebellum, it remains unclear how these two regions co-function. Here, we recorded neuronal activity in the output of cerebellum and basal ganglia, the cerebellar nuclei and substantia nigra pars reticulata, respectively, from ChR2 transgenic rats with high-density Neuropixels probes while optogenetically stimulating the cerebral cortex point-by-point. The temporal response patterns could be categorized into two classes in both cerebellar nuclei and substantia nigra pars reticulata. Among them, the fast excitatory response of the cerebellar nuclei due to the input of mossy fibers and the inhibitory response of the substantia nigra pars reticulata via the direct pathway were synchronized. This coincidence, reproduced in a spiking network simulation based on connectome data, was expected to synchronously activate the cerebral cortex via the thalamus. To further investigate the significance of this synchronous positive feedback, we constructed a reservoir model that mimics the time course of the activity dynamics of cerebral cortex and temporal responses of cerebellar nuclei and substantia nigra pars reticulata. Plasticity of both parallel fiber inputs to Purkinje cell and corticostriatal synapses onto the striatal neurons of the direct pathway was essential for successful learning of a reinforcement learning task. Notably, learning was inhibited when the timing of the cerebellar or basal ganglia output was delayed from the real data by 10 ms; the larger this delay, the slower the learning rate. This necessary temporal precision was observed only when the cerebral cortex operated in the β-to-γ frequency range. These results indicate that coordinated output of the cerebellum and basal ganglia, with input from the cerebral cortex in a narrow frequency band, facilitates brain-wide synergistic reinforcement learning. Thus, our findings contribute to a holistic understanding of the interactions among the cerebellum, basal ganglia, and cerebral cortex.

Population coding in the cerebellum: a machine learning perspective

A cerebellar population coding model for sensorimotor learning

Spike-based population coding and working memory

Population Rate Coding in Recurrent Neuronal Networks with Unreliable Synapses

A Hierarchically Organized Memory Model With Temporal Population Coding

Recurrent Circuit Based Neural Population Codes for Stimulus Representation and Inference

Localization of long-term synaptic plasticity defects in cerebellar circuits using optokinetic reflex learning profile

A Hierarchical Organized Memory Model Using Spiking Neurons

High-Fidelity Coding with Correlated Neurons

Beyond spiking networks: The computational advantages of dendritic amplification and input segregation

Clustering of Neural Activity: A Design Principle for Population Codes

Spiking network model of the cerebellum as a reinforcement learning machine

A Sparsely Distributed and Statistically Independent Representation of Parallel Fiber Coding in Cerebellum

Population coding in mouse visual cortex: response reliability and dissociability of stimulus tuning and noise correlation

Neuronal sequences in population bursts encode information in human cortex

Synergistic reinforcement learning by cooperation of the cerebellum and basal ganglia

Maximizing Information in Neuron Populations for Neuromorphic Spike Encoding

Deep Predictive Learning in Neocortex and Pulvinar

Sparse coding with a somato-dendritic rule

Heterogeneous encoding of temporal stimuli in the cerebellar cortex

CerebelluMorphic: Large-Scale Neuromorphic Model and Architecture for Supervised Motor Learning