Abstract:Several studies demonstrate that the structure of the brain increases in hierarchical complexity throughout development. We tested if the structure of artificial neural networks also increases in hierarchical complexity while learning a developing task, called the balance beam problem. Previous simulations of this developmental task do not reflect a necessary premise underlying development: a more complex structure can be built out of less complex ones, while ensuring that the more complex structure does not replace the less complex one. In order to address this necessity, we segregated the input set by subsets of increasing Orders of Hierarchical Complexity. This is a complexity measure that has been extensively shown to underlie the complexity behavior and hypothesized to underlie the complexity of the neural structure of the brain. After segregating the input set, minimal neural network models were trained separately for each input subset, and adjacent complexity models were analyzed sequentially to observe whether there was a structural progression. Results show that three different network structural progressions were found, performing with similar accuracy, pointing towards self-organization. Also, more complex structures could be built out of less complex ones without substituting them, successfully addressing catastrophic forgetting and leveraging performance of previous models in the literature. Furthermore, the model structures trained on the two highest complexity subsets performed better than simulations of the balance beam present in the literature. As a major contribution, this work was successful in addressing hierarchical complexity structural growth in neural networks, and is the first that segregates inputs by Order of Hierarchical Complexity. Since this measure can be applied to all domains of data, the present method can be applied to future simulations, systematizing the simulation of developmental and evolutionary structural growth in neural networks.

Neural networks grown and self-organized by noise

Self-organization of multi-layer spiking neural networks

Towards Self-Assembling Artificial Neural Networks through Neural Developmental Programs

Exploration Of A Mechanism To Form Bionic, Self-Growing And Self-Organizing Neural Network

Evolving Self-Assembling Neural Networks: From Spontaneous Activity to Experience-Dependent Learning

Growing Artificial Neural Networks for Control: the Role of Neuronal Diversity

HyperNCA: Growing Developmental Networks with Neural Cellular Automata

Research on Algorithms of Neural Network Based on Self-Growing of Neuron

Learning About Growing Neural Cellular Automata

Self-Organization of Spiking Neural Network Generating Autonomous Behavior in a Real Mobile Robot

Data-driven emergence of convolutional structure in neural networks

Towards New Generation, Biologically Plausible Deep Neural Network Learning

Neocognitron: a self organizing neural network model for a mechanism of pattern recognition unaffected by shift in position

Self-building Neural Networks

Hierarchical growth in neural networks structure: Organizing inputs by Order of Hierarchical Complexity

Evolving Connections in Group of Neurons for Robust Learning

Developing neuronal networks: Self-organized criticality predicts the future

Exploration of bionic self-growing and self-organizing neuron network

Neural Cellular Automata Can Respond to Signals

Continual Learning with Deep Artificial Neurons