Abstract:Biological neural networks are characterized by their high degree of plasticity, a core property that enables the remarkable adaptability of natural organisms. Importantly, this ability affects both the synaptic strength and the topology of the nervous systems. Artificial neural networks, on the other hand, have been mainly designed as static, fully connected structures that can be notoriously brittle in the face of changing environments and novel inputs. Building on previous works on Neural Developmental Programs (NDPs), we propose a class of self-organizing neural networks capable of synaptic and structural plasticity in an activity and reward-dependent manner which we call Lifelong Neural Developmental Program (LNDP). We present an instance of such a network built on the graph transformer architecture and propose a mechanism for pre-experience plasticity based on the spontaneous activity of sensory neurons. Our results demonstrate the ability of the model to learn from experiences in different control tasks starting from randomly connected or empty networks. We further show that structural plasticity is advantageous in environments necessitating fast adaptation or with non-stationary rewards.

What problem does this paper attempt to address?

The core issue this paper attempts to address is: how to design a self-organizing neural network with synaptic and structural plasticity that can start from random connections or an empty network and adaptively develop into a functional neural network through an experience-dependent learning process across different tasks. Specifically, the paper proposes a method called "Lifelong Neural Development Program" (LNDP) aimed at overcoming the vulnerability and lack of continuous learning ability of existing neural networks when faced with changing environments. ### Main Research Background 1. **Plasticity of Biological Neural Networks**: Biological neural networks exhibit a high degree of plasticity, which allows organisms in nature to quickly adapt to environmental changes in a short period. This plasticity affects not only synaptic strength but also the topology of the nervous system. 2. **Limitations of Artificial Neural Networks**: Most existing artificial neural networks (ANNs) are designed as static, fully connected structures, making them very vulnerable when faced with changing environments and new inputs, and unable to perform continuous learning. 3. **Insufficiencies of Existing Research**: Although some studies have attempted to improve the adaptability of ANNs through online learning and meta-learning, these methods usually rely on offline learning and cannot achieve lifelong learning. ### Main Contributions of the Paper 1. **Proposing the LNDP Framework**: The paper proposes a self-organizing neural network based on a graph transformer architecture, called the "Lifelong Neural Development Program" (LNDP). LNDP can adjust synaptic and structural plasticity throughout its lifecycle, enabling continuous learning. 2. **Introducing a Spontaneous Activity Mechanism**: The paper introduces a mechanism based on spontaneous activity (SA), allowing the network to undergo pre-experiential development even without environmental feedback. This mechanism helps the network self-organize in the early stages, forming initial functional structures. 3. **Validating the Effectiveness of LNDP**: The paper validates the effectiveness and advantages of LNDP in different environments through experiments on multiple control tasks (such as Cartpole, Acrobot, Pendulum, and Foraging). Particularly, in environments requiring rapid adaptation or non-stationary rewards, LNDP outperforms models without structural plasticity. ### Experimental Results 1. **In the Cartpole Task**: LNDP can develop a functional network from a randomly connected network in a short time and demonstrates rapid adaptation capabilities. Especially in the initial stages, LNDP can achieve higher rewards more quickly through the spontaneous activity mechanism. 2. **In the Foraging Task**: LNDP shows better adaptability in non-stationary reward environments, with its average fitness significantly higher than models without structural plasticity. 3. **In Other Tasks**: LNDP also shows certain advantages in the Acrobot and Pendulum tasks, especially in continuous control tasks, where only LNDP can find solutions in some runs. ### Conclusion By proposing the LNDP framework, the paper successfully addresses the vulnerability and lack of continuous learning ability of existing neural networks when faced with changing environments. LNDP can not only adaptively develop into functional networks from random connections or empty networks through an experience-dependent learning process across different tasks but also shows better adaptability in non-stationary reward environments. These results provide new insights for developing more flexible and adaptive artificial intelligence systems.

Evolving Self-Assembling Neural Networks: From Spontaneous Activity to Experience-Dependent Learning

Towards Self-Assembling Artificial Neural Networks through Neural Developmental Programs

Self-organization of multi-layer spiking neural networks

Born to Learn: the Inspiration, Progress, and Future of Evolved Plastic Artificial Neural Networks

Adaptive behavior with stable synapses

Incorporating structural plasticity into self-organization recurrent networks for sequence learning

Self-organization of microcircuits in networks of neurons with plastic synapses

Computational Modeling of Neural Plasticity for Self-Organization of Neural Networks

Learning the Plasticity: Plasticity-Driven Learning Framework in Spiking Neural Networks

Rewiring Neurons in Non-Stationary Environments

Neural networks grown and self-organized by noise

Evolving interpretable plasticity for spiking networks

Contrastive-Signal-Dependent Plasticity: Self-Supervised Learning in Spiking Neural Circuits

Evolution of Neural Development

Continual Learning with Deep Artificial Neurons

Growing Artificial Neural Networks for Control: the Role of Neuronal Diversity

Incremental Self-Organization of Spatio-Temporal Spike Pattern Detection

Neuroplastic Expansion in Deep Reinforcement Learning

Balancing complexity, performance and plausibility to meta learn plasticity rules in recurrent spiking networks.

Critical neural networks with short and long term plasticity

Emergent Structures and Lifetime Structure Evolution in Artificial Neural Networks