Abstract:Humans excel at adapting perceptions and actions to diverse environments, enabling efficient interaction with the external world. This adaptive capability relies on the biological nervous system (BNS), which activates different brain regions for distinct tasks. Meta-learning similarly trains machines to handle multiple tasks but relies on a fixed network structure, not as flexible as BNS. To investigate the role of flexible network structure (FNS) in meta-learning, we conduct extensive empirical and theoretical analyses, finding that model performance is tied to structure, with no universally optimal pattern across tasks. This reveals the crucial role of FNS in meta-learning, ensuring meta-learning to generate the optimal structure for each task, thereby maximizing the performance and learning efficiency of meta-learning. Motivated by this insight, we propose to define, measure, and model FNS in meta-learning. First, we define that an effective FNS should possess frugality, plasticity, and sensitivity. Then, to quantify FNS in practice, we present three measurements for these properties, collectively forming the \emph{structure constraint} with theoretical supports. Building on this, we finally propose Neuromodulated Meta-Learning (NeuronML) to model FNS in meta-learning. It utilizes bi-level optimization to update both weights and structure with the structure constraint. Extensive theoretical and empirical evaluations demonstrate the effectiveness of NeuronML on various tasks. Code is publicly available at \href{<a class="link-external link-https" href="https://github.com/WangJingyao07/NeuronML" rel="external noopener nofollow">this https URL</a>}{<a class="link-external link-https" href="https://github.com/WangJingyao07/NeuronML" rel="external noopener nofollow">this https URL</a>}.

Neuromodulated Meta-Learning

Neural Metamorphosis

Learning to Learn by Jointly Optimizing Neural Architecture and Weights

Connection-Adaptive Meta-Learning

Meta-Learning Strategies through Value Maximization in Neural Networks

Meta Neural Coordination

Brain-inspired Global-Local Learning Incorporated with Neuromorphic Computing

Structured flexibility in recurrent neural networks via neuromodulation

Lifelong Reinforcement Learning via Neuromodulation

A Meta Understanding of Meta-Learning

Meta-Modulation: A General Learning Framework for Cross-Task Adaptation

Meta-GF: Training Dynamic-Depth Neural Networks Harmoniously.

Meta-Learned Models of Cognition

Data-Efficient Brain Connectome Analysis via Multi-Task Meta-Learning

Meta Module Network for Compositional Visual Reasoning

Meta ControlNet: Enhancing Task Adaptation via Meta Learning

Random Graph-Based Neuromorphic Learning with a Layer-Weaken Structure

Learning to Modulate Random Weights: Neuromodulation-inspired Neural Networks For Efficient Continual Learning

Learning Large-scale Neural Fields via Context Pruned Meta-Learning

Meta-learning, social cognition and consciousness in brains and machines