Abstract:In biological evolution complex neural structures grow from a handful of cellular ingredients. As genomes in nature are bounded in size, this complexity is achieved by a growth process where cells communicate locally to decide whether to differentiate, proliferate and connect with other cells. This self-organisation is hypothesized to play an important part in the generalisation, and robustness of biological neural networks. Artificial neural networks (ANNs), on the other hand, are traditionally optimized in the space of weights. Thus, the benefits and challenges of growing artificial neural networks remain understudied. Building on the previously introduced Neural Developmental Programs (NDP), in this work we present an algorithm for growing ANNs that solve reinforcement learning tasks. We identify a key challenge: ensuring phenotypic complexity requires maintaining neuronal diversity, but this diversity comes at the cost of optimization stability. To address this, we introduce two mechanisms: (a) equipping neurons with an intrinsic state inherited upon neurogenesis; (b) lateral inhibition, a mechanism inspired by biological growth, which controlls the pace of growth, helping diversity persist. We show that both mechanisms contribute to neuronal diversity and that, equipped with them, NDPs achieve comparable results to existing direct and developmental encodings in complex locomotion tasks

What problem does this paper attempt to address?

The core problem that this paper attempts to solve is: **How to ensure that artificial neural networks (ANNs) can exhibit complex behaviors during growth and successfully solve reinforcement learning (RL) tasks by maintaining neuronal diversity**. ### Background and Problem Traditional artificial neural networks usually directly optimize weights, while biological neural networks form complex structures through cell proliferation, differentiation, and connection. This self - organizing process gives biological neural networks good generalization ability and robustness. However, artificial neural networks face a challenge during growth: if all neurons differentiate into the same type, the network will become homogeneous and unable to exhibit complex behaviors. Therefore, how to maintain neuronal diversity during the growth of artificial neural networks has become a key issue. ### Main Contributions To solve the above problems, the author proposes two mechanisms: 1. **Intrinsic States**: - Each neuron inherits the intrinsic states of its parent when it is generated, and these states will not change during the growth process. This ensures that there is a certain amount of diversity initially, and as the network grows, these unique states will be copied to new neurons, thus maintaining diversity. 2. **Lateral Inhibition**: - When a neuron differentiates, generates new neurons, or updates connections, it will inhibit its neighboring neurons from performing similar operations in the following several steps. This mechanism prevents all neurons from making the same decision simultaneously, thus helping to maintain neuronal diversity. ### Experimental Results Through experimental verification, these two mechanisms significantly improve the performance of neural networks, especially in complex motion - control tasks. Specifically: - **Intrinsic States**: After the introduction of intrinsic states, NDP (Neural Developmental Programs) can find the optimal strategy in all test tasks, while without intrinsic states, the performance of NDP is very poor and cannot even be improved. - **Lateral Inhibition**: Lateral inhibition helps to maintain neuronal diversity throughout the evolution and development process. Experimental results show that without lateral inhibition, neuronal diversity will disappear rapidly in the early stage. ### Summary By introducing the intrinsic state and lateral inhibition mechanisms, this paper solves the homogeneity problem that is prone to occur in the growth process of artificial neural networks, and successfully enables NDP to perform excellently in complex 3D motion and manipulation tasks. This shows the importance of neuronal diversity in neural network growth and provides new ideas for future research.

Growing Artificial Neural Networks for Control: the Role of Neuronal Diversity