Abstract:Training a large set of machine learning algorithms to convergence in order to select the best-performing algorithm for a dataset is computationally wasteful. Moreover, in a budget-limited scenario, it is crucial to carefully select an algorithm candidate and allocate a budget for training it, ensuring that the limited budget is optimally distributed to favor the most promising candidates. Casting this problem as a Markov Decision Process, we propose a novel framework in which an agent must select in the process of learning the most promising algorithm without waiting until it is fully trained. At each time step, given an observation of partial learning curves of algorithms, the agent must decide whether to allocate resources to further train the most promising algorithm (exploitation), to wake up another algorithm previously put to sleep, or to start training a new algorithm (exploration). In addition, our framework allows the agent to meta-learn from learning curves on past datasets along with dataset meta-features and algorithm hyperparameters. By incorporating meta-learning, we aim to avoid myopic decisions based solely on premature learning curves on the dataset at hand. We introduce two benchmarks of learning curves that served in international competitions at WCCI'22 and AutoML-conf'22, of which we analyze the results. Our findings show that both meta-learning and the progression of learning curves enhance the algorithm selection process, as evidenced by methods of winning teams and our DDQN baseline, compared to heuristic baselines or a random search. Interestingly, our cost-effective baseline, which selects the best-performing algorithm w.r.t. a small budget, can perform decently when learning curves do not intersect frequently.

GreedyAgent: Crafting Efficient Agents for Meta-learning from Learning Curves Via Greedy Algorithm Selection

Meta-Learning from Learning Curves for Budget-Limited Algorithm Selection

A Dynamically Adaptive Approach to Reducing Strategic Interference for Multi-agent Systems

Dueling Network Architecture for Multi-Agent Deep Deterministic Policy Gradient

Adaptive Submodular Meta-Learning

Adaptive Incentive Design with Multi-Agent Meta-Gradient Reinforcement Learning

AG-Meta: Adaptive graph meta-learning via representation consistency over local subgraphs

Fast Adaptation Via Meta Learning in Multi-Agent Cooperative Tasks

Robust Fast Adaptation from Adversarially Explicit Task Distribution Generation

MAML2: meta reinforcement learning via meta-learning for task categories

Continuous Self-Adaptive Optimization to Learn Multi-Task Multi-Agent

Curriculum in Gradient-Based Meta-Reinforcement Learning

A Meta Multi-agent Reinforcement Learning Algorithm for Multi-intersection Traffic Signal Control

Unsupervised Meta-Learning for Reinforcement Learning

Model-based Adversarial Meta-Reinforcement Learning

Fairness-Aware Meta-Learning via Nash Bargaining

Online meta-learning by parallel algorithm competition

A Theoretical Understanding of Gradient Bias in Meta-Reinforcement Learning

Learning to Learn by Jointly Optimizing Neural Architecture and Weights

Meta-Learning Transferable Active Learning Policies by Deep Reinforcement Learning

Meta-learning curiosity algorithms