Abstract:This work proposes a new perspective on the structure of Neural Networks (NNs). Traditional Neural Networks are typically tree-like structures for convenience, which can be predefined or learned by NAS methods. However, such a structure can not facilitate communications between nodes at the same level or signal transmissions to previous levels. These defects prevent effective collaboration, restricting the capabilities of neural networks. It is well-acknowledged that the biological neural system contains billions of neural units. Their connections are far more complicated than the current NN structure. To enhance the representational ability of neural networks, existing works try to increase the depth of the neural network and introduce more parameters. However, they all have limitations with constrained parameters. In this work, we introduce a synchronous graph-based structure to establish a novel way of organizing the neural units: the Neural Modules. This framework allows any nodes to communicate with each other and encourages neural units to work collectively, demonstrating a departure from the conventional constrained paradigm. Such a structure also provides more candidates for the NAS methods. Furthermore, we also propose an elegant regularization method to organize neural units into multiple independent, balanced neural modules systematically. This would be convenient for handling these neural modules in parallel. Compared to traditional NNs, our method unlocks the potential of NNs from tree-like structures to general graphs and makes NNs be optimized in an almost complete set. Our approach proves adaptable to diverse tasks, offering compatibility across various scenarios. Quantitative experimental results substantiate the potential of our structure, indicating the improvement of NNs.

A Study of Functions Distribution of Neural Networks

Adaptive Neural Networks Control and Learning for Overhead Crane Systems

A Nonlinear Adaptive Controller Based on RBF Networks

CMNN: Coupled Modular Neural Network.

Reinforcement Learning-Based Control for Nonlinear Discrete-Time Systems with Unknown Control Directions and Control Constraints

Broad-learning recurrent Hermite neural control for unknown nonlinear systems

A New Neural Network Model And Its Application

Tree-Like Branching Network for Multi-class Classification

Delay-independent and delay-dependent stability of a novel delayed neural network

Hermite broad-learning recurrent neural control with adaptive learning rate for nonlinear systems

The Outputs Robustness of Boolean Control Networks Via Pinning Control

Novel neural controller for robot manipulator

Neural Network Verification with Branch-and-Bound for General Nonlinearities

Event-Triggered Neural-Network Adaptive Control for Strict-Feedback Nonlinear Systems: Selections on Valid Compact Sets

Automatic Organization of Neural Modules for Enhanced Collaboration in Neural Networks

ReachNN: Reachability Analysis of Neural-Network Controlled Systems.

A Theoretical Study of The Relationship Between Whole An ELM Network and Its Subnetworks

Network Controllability Robustness Learning Via Spatial Graph Neural Networks

Biology-inspired joint distribution neurons based on Hierarchical Correlation Reconstruction allowing for multidirectional neural networks

A Multilayer Feed Forward Small-World Neural Network Controller and Its Application on Electrohydraulic Actuation System.

Coordinative Control of Excitation, Fast Valving and Resistant Braking Using Artificial Neural Network