Abstract:Neurally inspired robotics already has a long history that includes reactive systems emulating reflexes, neural oscillators to generate movement patterns, and neural networks as trainable filters for high-dimensional sensory information. Neural inspiration has been less successful at the level of cognition. Decision-making, planning, building and using memories, for instance, are more often addressed in terms of computational algorithms than through neural process models. To move neural process models beyond reactive behavior toward cognition, the capacity to autonomously generate sequences of processing steps is critical. We review a potential solution to this problem that is based on strongly recurrent neural networks described as neural dynamic systems. Their stable states perform elementary motor or cognitive functions while coupled to sensory inputs. The state of the neural dynamics transitions to a new motor or cognitive function when a previously stable neural state becomes unstable. Only when a neural robotic system is capable of acting autonomously does it become a useful to a human user. We demonstrate how a neural dynamic architecture that supports autonomous sequence generation can engage in such interaction. A human user presents colored objects to the robot in a particular order, thus defining a serial order of color concepts. The user then exposes the system to a visual scene that contains the colored objects in a new spatial arrangement. The robot autonomously builds a scene representation by sequentially bringing objects into the attentional foreground. Scene memory updates if the scene changes. The robot performs visual search and then reaches for the objects in the instructed serial order. In doing so, the robot generalizes across time and space, is capable of waiting when an element is missing, and updates its action plans online when the scene changes. The entire flow of behavior emerges from a time-continuous neural dynamics without any controlling or supervisory algorithm.

Autonomous reinforcement of behavioral sequences in neural dynamics

Sequential Action-Induced Invariant Representation for Reinforcement Learning

Deep Reinforcement Learning for the Biologically Inspired Social Behaviour of Autonomous Robots Acting in Dynamic Environments

Reinforcement Learning in a Neurally Controlled Robot Using Dopamine Modulated STDP

Autonomous Sequence Generation for a Neural Dynamic Robot: Scene Perception, Serial Order, and Object-Oriented Movement

Image-Based Deep Reinforcement Learning with Intrinsically Motivated Stimuli: On the Execution of Complex Robotic Tasks

A Neuromorphic Architecture for Reinforcement Learning from Real-Valued Observations

Inferring Time-Varying Internal Models of Agents Through Dynamic Structure Learning

Multi-Sacle Dynamic Coding Improved Spiking Actor Network for Reinforcement Learning

Population-coding and Dynamic-neurons improved Spiking Actor Network for Reinforcement Learning

Toward robust and scalable deep spiking reinforcement learning

DynaSTI: Dynamics Modeling with Sequential Temporal Information for Reinforcement Learning in Atari

Neuro-Inspired Reinforcement Learning to Improve Trajectory Prediction in Reward-Guided Behavior

Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learning.

Dynamics Adaptive Safe Reinforcement Learning with a Misspecified Simulator

Neural Modulation for Reinforcement Learning in Developmental Networks Facing an Exponential No. of States

End-to-End Robotic Reinforcement Learning without Reward Engineering

Dynamic Sparse Training for Deep Reinforcement Learning

Learning Movement Sequences with a Delayed Reward Signal in a Hierarchical Model of Motor Function.

Dense Dynamics-Aware Reward Synthesis: Integrating Prior Experience with Demonstrations

R-SARL: Crowd-aware Navigation Based Deep Reinforcement Learning for Nonholonomic Robot in Complex Environments