Abstract:The cerebellum is known to be intimately involved in motor control and learning. The most distinctive characteristics of the cerebellum is its anatomically well-defined uniform neuronal circuitry over the cerebellar cortex and the physiologically identified synaptic plasticity driven by ultra-low firing climbing fiber input to Purkinje cells [1]. To date, realistic spiking neuron models and more abstract neural network models of the cerebellum have been constructed to argue roles of each neuron type, namely granular (Gr), Golgi (Go), basket (Ba), stellate (St), and Purkinje (Pk) cells that form the uniform cerebellar cortical neuronal circuitry [2]. However not much attention has been made on the bi-hemispheric structure of the cerebellum in relation to the ultra-low firing climbing fiber input [3]. Here we configured a cerebellar neuronal network model consisting of two hemispheres that have the identical anatomically plausible connections of each neuron type. One and only difference between the models of the two hemispheres is in their climbing fiber input. For example, the amplitude of the climbing fiber input to the right cerebellum increases with behavioral errors in one direction while that to the left cerebellum simultaneously decreases only slightly and is clipped at zero due to the ultra-low firing rate. In this way, even we implemented the identical synaptic plasticity (LTP and LTD) at Gr and Pk cells for the left and right hemisphere models, their output (Pk cell activities) would result in asymmetry. We determined the number of each neuron type as follows so that the model can run in real time and be applied to adaptive control of an unstable robot: 755 Gr, 5 Go, 5 Ba/St, 1Pk in each hemisphere. We chose a 2-wheeled unstable robot (e-nuvo, ZMP) that naturally shows asymmetry in its forward and backward motion. A common feedback controller (PD) was used in parallel with the cerebellar model to employ the feedback error learning scheme in which the model induces the synaptic plasticity to reduce the output of the PD. In real world testing, the proposed model was able to control the robot to follow a single sinusoidal trajectory (0.1 Hz), and also managed to compensate for an abrupt imposition of extra-weight placed either on-centered (balance load) or off-centered (unbalance load) on the robot body. During this adaptive process, the bi-hemispheric cerebellar model gradually took over the contribution of the PD controller by modifying their adaptive synaptic weights so that each hemisphere can optimize the forward and backward motion of the robot. Furthermore, we tested the controller lacking the realistic cf input using a learning rule without considering the ultra-low spontaneous dc firing. Even in such a case, the proposed controller successfully compensated for the balanced or the unbalanced load initially, however, it became gradually unstable as the experiment continued. These results suggest that the bi-hemispheric cerebellar structure may play a role in directionally asymmetrical motor control and learning, while the cf spontaneous activity may serve for stabilization of motor learning.

A differential Hebbian framework for biologically-plausible motor control

Self-configuring feedback loops for sensorimotor control

An in-silico framework for modeling optimal control of neural systems

Online Reinforcement Learning-based Neural Network Controller Design for Affine Nonlinear Discrete-time Systems.

Motion Control Of A Snake Robot Via Cerebellum-Inspired Learning Control

Neuromimetic Dynamic Networks with Hebbian Learning

Model-Free Robust Optimal Feedback Mechanisms of Biological Motor Control

Feedback control of recurrent dynamics constrains learning timescales during motor adaptation

Construction of a bi-hemispheric cerebellar neuronal network model with realistic climbing fiber input and its application to adaptive control of an unstable robot

Learning from Neural Control in Motor Systems.

RLOC: Neurobiologically Inspired Hierarchical Reinforcement Learning Algorithm for Continuous Control of Nonlinear Dynamical Systems

Brain-like neural dynamics for behavioral control develop through reinforcement learning

A modular framework for stabilizing deep reinforcement learning control

Synthesizing Neural Network Controllers with Probabilistic Model based Reinforcement Learning

Neuromimetic Control -- A Linear Model Paradigm

Hierarchical motor control in mammals and machines

Learning cortical hierarchies with temporal Hebbian updates

Adaptive Robotic Control Driven by a Versatile Spiking Cerebellar Network

Learning to control non-equilibrium dynamics using local imperfect gradients

Cerebellar-Inspired Learning Rule for Gain Adaptation of Feedback Controllers

A Revised Computational Neuroanatomy for Motor Control