Abstract:Several studies demonstrate that the structure of the brain increases in hierarchical complexity throughout development. We tested if the structure of artificial neural networks also increases in hierarchical complexity while learning a developing task, called the balance beam problem. Previous simulations of this developmental task do not reflect a necessary premise underlying development: a more complex structure can be built out of less complex ones, while ensuring that the more complex structure does not replace the less complex one. In order to address this necessity, we segregated the input set by subsets of increasing Orders of Hierarchical Complexity. This is a complexity measure that has been extensively shown to underlie the complexity behavior and hypothesized to underlie the complexity of the neural structure of the brain. After segregating the input set, minimal neural network models were trained separately for each input subset, and adjacent complexity models were analyzed sequentially to observe whether there was a structural progression. Results show that three different network structural progressions were found, performing with similar accuracy, pointing towards self-organization. Also, more complex structures could be built out of less complex ones without substituting them, successfully addressing catastrophic forgetting and leveraging performance of previous models in the literature. Furthermore, the model structures trained on the two highest complexity subsets performed better than simulations of the balance beam present in the literature. As a major contribution, this work was successful in addressing hierarchical complexity structural growth in neural networks, and is the first that segregates inputs by Order of Hierarchical Complexity. Since this measure can be applied to all domains of data, the present method can be applied to future simulations, systematizing the simulation of developmental and evolutionary structural growth in neural networks.

Using Single-Neuron Representations for Hierarchical Concepts as Abstractions of Multi-Neuron Representations

Multi-Neuron Representations of Hierarchical Concepts in Spiking Neural Networks

Learning Hierarchically Structured Concepts

A Hierarchical Model for Structure Learning Based on the Physiological Characteristics of Neurons

HINT: Hierarchical Neuron Concept Explainer

Hierarchies of description enable understanding of cognitive phenomena in terms of neuron activity

Cause and Effect: Hierarchical Concept-based Explanation of Neural Networks

Syntactic vs Semantic Linear Abstraction and Refinement of Neural Networks

How to Represent Part-Whole Hierarchies in a Neural Network

Multilevel Interpretability Of Artificial Neural Networks: Leveraging Framework And Methods From Neuroscience

Unsupervised representation learning with Hebbian synaptic and structural plasticity in brain-like feedforward neural networks

A Mathematical Approach to Constraining Neural Abstraction and the Mechanisms Needed to Scale to Higher-Order Cognition

Hierarchical modularity in human brain functional networks

Hierarchical growth in neural networks structure: Organizing inputs by Order of Hierarchical Complexity

Learning cortical hierarchies with temporal Hebbian updates

Resolving Structure in Human Brain Organization: Identifying Mesoscale Organization in Weighted Network Representations

Modeling Hierarchical Brain Networks Via Volumetric Sparse Deep Belief Network

A hierarchy index for networks in the brain reveals a complex entangled organizational structure

Hierarchical organization in neuronal functional networks during working memory tasks

Hierarchical Organization of Functional Brain Networks Revealed by Hybrid Spatiotemporal Deep Learning

Multilayer adaptive networks in neuronal processing