Abstract:The autonomous learning of new goals in robotics remains a complex issue to address. Here, we propose a model where curiosity influence learning flexibility. To do so, this paper proposes to root curiosity and attention together by taking inspiration from the Locus Coeruleus-Norepinephrine system along with various cognitive processes such as cognitive persistence and visual habituation. We apply our approach by experimenting with a simulated robotic arm on a set of objects with varying difficulty. The robot first discovers new goals via bottom-up attention through motor babbling with an inhibition of return mechanism, then engage to the learning of goals due to neural activity arising within the curiosity mechanism. The architecture is modelled with dynamic neural fields and the learning of goals such as pushing the objects in diverse directions is supported by the use of forward and inverse models implemented by multi-layer perceptrons. The adoption of dynamic neural fields to model curiosity, habituation and persistence allows the robot to demonstrate various learning trajectories depending on the object. In addition, the approach exhibits interesting properties regarding the learning of similar goals as well as the continuous switch between exploration and exploitation.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the complex problem of autonomous goal learning for robots. Specifically, the author proposes a new model to enhance the flexibility of learning by introducing curiosity. The core of this method is to combine curiosity and the attention mechanism, which is inspired by the Locus Coeruleus - Norepinephrine system (LC - NE) in the brain, as well as various cognitive processes such as cognitive persistence and visual habituation. #### Main problems and solutions 1. **Goal discovery and learning**: - **Problem**: How can a robot autonomously discover new goals and learn to interact with these goals? - **Solution**: The author proposes an architecture based on Dynamic Neural Fields (DNF), which can adaptively adjust the robot's behavior according to the stimuli in the environment. The robot first performs goal discovery through the bottom - up attention mechanism, and then participates in goal learning through the neural activities generated by the curiosity mechanism. 2. **Balance between exploration and exploitation**: - **Problem**: How to find a balance between exploring new goals and exploiting known goals? - **Solution**: The two activation modes (tonic and phasic) of the LC - NE system correspond to exploration and exploitation respectively. In the exploration phase, the robot shows a high arousal level (tonic activation), while in the exploitation phase, it shows task - oriented behavior (phasic activation). This switching depends on the changes in learning progress and error, thus achieving a dynamic balance between exploration and exploitation. 3. **Continuous learning and forgetting**: - **Problem**: How can a robot avoid over - focusing on old goals while learning new goals? - **Solution**: By introducing the habituation mechanism, the robot can gradually reduce its attention to familiar goals, so that it can focus more on new goals. In addition, the cognitive persistence mechanism ensures that the robot will not give up easily when facing difficult tasks. 4. **Multimodal perception and action control**: - **Problem**: How to effectively process visual information and the related action control? - **Solution**: The perception module processes the color and motion angle of objects through a 2D neural field and combines Dynamic Movement Primitives (DMP) to achieve action control. This enables the robot to flexibly respond to different goals in a complex environment. ### Summary This paper proposes a novel robot cognitive architecture by integrating research in multiple aspects such as curiosity, attention, LC - NE system, and dynamic neural fields, to solve the problem of robots autonomously discovering and learning new goals in a complex environment. This method not only improves the learning flexibility of robots but also shows the ability to effectively switch between exploration and exploitation, providing new ideas and methods for future robot research. --- If you have more questions or need further assistance, please feel free to let me know!

Dynamic Neural Curiosity Enhances Learning Flexibility for Autonomous Goal Discovery

Active Learning of Inverse Models with Intrinsically Motivated Goal Exploration in Robots

Prototyping three key properties of specific curiosity in computational reinforcement learning

Meta-learning curiosity algorithms

Intrinsic Motivation and Episodic Memories for Robot Exploration of High-Dimensional Sensory Spaces

Curiosity-Driven Reinforcement Learning based Low-Level Flight Control

Novelty and habituation: the driving forces in early stage learning for developmental robotics

A unified strategy for implementing curiosity and empowerment driven reinforcement learning

A Curiosity Algorithm for Robots Based on the Free Energy Principle

A Neuro-Cognitive System and Its Application in Robotics

Computational Theories of Curiosity-Driven Learning

Cognitive Navigation by Neuro-Inspired Localization, Mapping, and Episodic Memory

Curiosity-Driven Exploration via Latent Bayesian Surprise

Language Grounding through Social Interactions and Curiosity-Driven Multi-Goal Learning

Curiosity driven reinforcement learning for motion planning on humanoids

Curiosity in Hindsight: Intrinsic Exploration in Stochastic Environments

An Adversarial Objective for Scalable Exploration

Learning robotic manipulation skills with multiple semantic goals by conservative curiosity-motivated exploration

Curiosity-driven Exploration by Self-supervised Prediction

Dynamic Memory-based Curiosity: A Bootstrap Approach for Exploration

Exploring unknown environments: motivated developmental learning for autonomous navigation of mobile robots