Abstract:A longstanding goal of artificial general intelligence is highly capable generalists that can learn from diverse experiences and generalize to unseen tasks. The language and vision communities have seen remarkable progress toward this trend by scaling up transformer-based models trained on massive datasets, while reinforcement learning (RL) agents still suffer from poor generalization capacity under such paradigms. To tackle this challenge, we propose Meta Decision Transformer (Meta-DT), which leverages the sequential modeling ability of the transformer architecture and robust task representation learning via world model disentanglement to achieve efficient generalization in offline meta-RL. We pretrain a context-aware world model to learn a compact task representation, and inject it as a contextual condition to the causal transformer to guide task-oriented sequence generation. Then, we subtly utilize history trajectories generated by the meta-policy as a self-guided prompt to exploit the architectural inductive bias. We select the trajectory segment that yields the largest prediction error on the pretrained world model to construct the prompt, aiming to encode task-specific information complementary to the world model maximally. Notably, the proposed framework eliminates the requirement of any expert demonstration or domain knowledge at test time. Experimental results on MuJoCo and Meta-World benchmarks across various dataset types show that Meta-DT exhibits superior few and zero-shot generalization capacity compared to strong baselines while being more practical with fewer prerequisites. Our code is available at <a class="link-external link-https" href="https://github.com/NJU-RL/Meta-DT" rel="external noopener nofollow">this https URL</a>.

Meta Distant Transfer Learning for Pre-trained Language Models.

Meta-Learning the Difference: Preparing Large Language Models for Efficient Adaptation

Meta-DT: Offline Meta-RL as Conditional Sequence Modeling with World Model Disentanglement

Meta Fine-Tuning Neural Language Models for Multi-Domain Text Mining

Meta-learning Transferable Representations with a Single Target Domain

Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains

Cross-Lingual Language Model Meta-Pretraining

Adaptive Meta-Domain Transfer Learning (AMDTL): A Novel Approach for Knowledge Transfer in AI

Towards Understanding Transfer Learning Algorithms Using Meta Transfer Features

Meta-Transfer Learning Through Hard Tasks

Task-Distributionally Robust Data-Free Meta-Learning.

Conditionally Adaptive Multi-Task Learning: Improving Transfer Learning in NLP Using Fewer Parameters & Less Data

MAML-en-LLM: Model Agnostic Meta-Training of LLMs for Improved In-Context Learning

Meta-RTL: Reinforcement-Based Meta-Transfer Learning for Low-Resource Commonsense Reasoning

Meta-LMTC - Meta-Learning for Large-Scale Multi-Label Text Classification.

MetaTool: Facilitating Large Language Models to Master Tools with Meta-task Augmentation

Pre-Training and Personalized Fine-Tuning via Over-the-Air Federated Meta-Learning: Convergence-Generalization Trade-Offs

Meta-Learning for Low-resource Natural Language Generation in Task-oriented Dialogue Systems.

When to Use Multi-Task Learning vs Intermediate Fine-Tuning for Pre-Trained Encoder Transfer Learning

MAML2: meta reinforcement learning via meta-learning for task categories