Abstract:Several studies demonstrate that the structure of the brain increases in hierarchical complexity throughout development. We tested if the structure of artificial neural networks also increases in hierarchical complexity while learning a developing task, called the balance beam problem. Previous simulations of this developmental task do not reflect a necessary premise underlying development: a more complex structure can be built out of less complex ones, while ensuring that the more complex structure does not replace the less complex one. In order to address this necessity, we segregated the input set by subsets of increasing Orders of Hierarchical Complexity. This is a complexity measure that has been extensively shown to underlie the complexity behavior and hypothesized to underlie the complexity of the neural structure of the brain. After segregating the input set, minimal neural network models were trained separately for each input subset, and adjacent complexity models were analyzed sequentially to observe whether there was a structural progression. Results show that three different network structural progressions were found, performing with similar accuracy, pointing towards self-organization. Also, more complex structures could be built out of less complex ones without substituting them, successfully addressing catastrophic forgetting and leveraging performance of previous models in the literature. Furthermore, the model structures trained on the two highest complexity subsets performed better than simulations of the balance beam present in the literature. As a major contribution, this work was successful in addressing hierarchical complexity structural growth in neural networks, and is the first that segregates inputs by Order of Hierarchical Complexity. Since this measure can be applied to all domains of data, the present method can be applied to future simulations, systematizing the simulation of developmental and evolutionary structural growth in neural networks.

Human Learning of Hierarchical Graphs

Human learning of hierarchical graphs

Towards Human-like Perception: Learning Structural Causal Model in Heterogeneous Graph

Humans can navigate complex graph structures acquired during latent learning

Computational evidence for hierarchically structured reinforcement learning in humans

Dynamic emergence of relational structure network in human brains

Time-resolved functional connectivity during visuomotor graph learning

Electrophysiological Signatures of Hierarchical Learning

A computational hierarchy in human cortex

Human Sensitivity to Community Structure Is Robust to Topological Variation

Compressed higher-order structures facilitate human knowledge network learning

Hierarchical structure and the prediction of missing links in networks

Long-horizon associative learning explains human sensitivity to statistical and network structures in auditory sequences

Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective

Seeing "what" Through "why": Evidence from Probing the Causal Structure of Hierarchical Motion.

A Hierarchical Graph Learning Model for Brain Network Regression Analysis.

Neural mechanisms for learning hierarchical structures of information

A reachable probability approach for the analysis of spatio-temporal dynamics in the human functional network

Hierarchical growth in neural networks structure: Organizing inputs by Order of Hierarchical Complexity

Explainable spatio-temporal graph evolution learning with applications to dynamic brain network analysis during development