Abstract:The cerebellum is involved in a large number of different neural processes, especially in associative learning and in fine motor control. To develop a comprehensive theory of sensorimotor learning and control, it is crucial to determine the neural basis of coding and plasticity embedded into the cerebellar neural circuit and how they are translated into behavioral outcomes in learning paradigms. Learning has to be inferred from the interaction of an embodied system with its real environment, and the same cerebellar principles derived from cell physiology have to be able to drive a variety of tasks of different nature, calling for complex timing and movement patterns. We have coupled a realistic cerebellar spiking neural network (SNN) with a real robot and challenged it in multiple diverse sensorimotor tasks. Encoding and decoding strategies based on neuronal firing rates were applied. Adaptive motor control protocols with acquisition and extinction phases have been designed and tested, including an associative Pavlovian task (Eye blinking classical conditioning), a vestibulo-ocular task and a perturbed arm reaching task operating in closed-loop. The SNN processed in real-time mossy fiber inputs as arbitrary contextual signals, irrespective of whether they conveyed a tone, a vestibular stimulus or the position of a limb. A bidirectional long-term plasticity rule implemented at parallel fibers-Purkinje cell synapses modulated the output activity in the deep cerebellar nuclei. In all tasks, the neurorobot learned to adjust timing and gain of the motor responses by tuning its output discharge. It succeeded in reproducing how human biological systems acquire, extinguish and express knowledge of a noisy and changing world. By varying stimuli and perturbations patterns, real-time control robustness and generalizability were validated. The implicit spiking dynamics of the cerebellar model fulfill timing, prediction and learning functions.

Neuronal Circuit Policies

Can a Compact Neuronal Circuit Policy be Re-purposed to Learn Simple Robotic Control?

Discovering neural policies to drive behaviour by integrating deep reinforcement learning agents with biological neural networks

Deep Reinforcement Learning for Neural Control

Neural Circuit Policies for Virtual Character Control

Neural Circuit Policies Imposing Visual Perceptual Autonomy

A Simplified Cerebellum-Based Model for Motor Control in Brain Based Devices.

Neural Circuit Architectural Priors for Embodied Control

Designing Worm-inspired Neural Networks for Interpretable Robotic Control

A Simplified Cerebellar Model with Priority-based Delayed Eligibility Trace Learning for Motor Control

A Neuromorphic Architecture for Reinforcement Learning from Real-Valued Observations

Rewiring Neurons in Non-Stationary Environments

Coprocessor Actor Critic: A Model-Based Reinforcement Learning Approach For Adaptive Brain Stimulation

Tuning Synaptic Connections Instead of Weights by Genetic Algorithm in Spiking Policy Network

NeuroCERIL: Robotic Imitation Learning via Hierarchical Cause-Effect Reasoning in Programmable Attractor Neural Networks

Spiking network model of the cerebellum as a reinforcement learning machine

Model-Based and Model-Free Replay Mechanisms for Reinforcement Learning in Neurorobotics

Memristive Circuit Implementation of Caenorhabditis Elegans Mechanism for Neuromorphic Computing.

Adaptive Robotic Control Driven by a Versatile Spiking Cerebellar Network

Brain-like neural dynamics for behavioral control develop through reinforcement learning

Active Predicting Coding: Brain-Inspired Reinforcement Learning for Sparse Reward Robotic Control Problems