Abstract:Contextual Self-Modulation (CSM) is a potent regularization mechanism for the Neural Context Flow (NCF) framework which demonstrates powerful meta-learning of physical systems. However, CSM has limitations in its applicability across different modalities and in high-data regimes. In this work, we introduce two extensions: $i$CSM, which expands CSM to infinite-dimensional tasks, and StochasticNCF, which improves scalability. These extensions are demonstrated through comprehensive experimentation on a range of tasks, including dynamical systems with parameter variations, computer vision challenges, and curve fitting problems. $i$CSM embeds the contexts into an infinite-dimensional function space, as opposed to CSM which uses finite-dimensional context vectors. StochasticNCF enables the application of both CSM and $i$CSM to high-data scenarios by providing an unbiased approximation of meta-gradient updates through a sampled set of nearest environments. Additionally, we incorporate higher-order Taylor expansions via Taylor-Mode automatic differentiation, revealing that higher-order approximations do not necessarily enhance generalization. Finally, we demonstrate how CSM can be integrated into other meta-learning frameworks with FlashCAVIA, a computationally efficient extension of the CAVIA meta-learning framework (Zintgraf et al. 2019). FlashCAVIA outperforms its predecessor across various benchmarks and reinforces the utility of bi-level optimization techniques. Together, these contributions establish a robust framework for tackling an expanded spectrum of meta-learning tasks, offering practical insights for out-of-distribution generalization. Our open-sourced library, designed for flexible integration of self-modulation into contextual meta-learning workflows, is available at \url{<a class="link-external link-http" href="http://github.com/ddrous/self-mod" rel="external noopener nofollow">this http URL</a>}.

Magnetism of epitaxial bcc iron on Ag(001) observed by spin-polarized photoemission.

Meta-Reinforcement Learning Robust to Distributional Shift Via Performing Lifelong In-Context Learning

Predictive value of a positive exercise stress testing and correlations with cardiovascular risk factors.

Meta-Reinforcement Learning with Dynamic Adaptiveness Distillation

Reinforcement Learning with Brain-Inspired Modulation can Improve Adaptation to Environmental Changes

Learn to Effectively Explore in Context-Based Meta-RL

MetaCURE: Meta Reinforcement Learning with Empowerment-Driven Exploration

Exploration With Task Information for Meta Reinforcement Learning

NoRML: No-Reward Meta Learning

Data-Efficient Task Generalization via Probabilistic Model-based Meta Reinforcement Learning

Learning to reinforcement learn for Neural Architecture Search

Extending Contextual Self-Modulation: Meta-Learning Across Modalities, Task Dimensionalities, and Data Regimes

Constrained Meta Agnostic Reinforcement Learning

Guided Meta-Policy Search

Efficient Meta Reinforcement Learning for Preference-based Fast Adaptation

Enhancing Context-Based Meta-Reinforcement Learning Algorithms Via An Efficient Task Encoder (Student Abstract)

Meta-Reinforcement Learning in Nonstationary and Nonparametric Environments

Lifelong Reinforcement Learning via Neuromodulation

Meta-Reinforcement Learning Using Model Parameters

Meta-reinforcement learning via orbitofrontal cortex

Dream to Adapt: Meta Reinforcement Learning by Latent Context Imagination and MDP Imagination