Abstract:Low-resource automatic speech recognition (ASR) is challenging, as the low-resource target language data cannot well train an ASR model. To solve this issue, meta-learning formulates ASR for each source language into many small ASR tasks and meta-learns a model initialization on all tasks from different source languages to access fast adaptation on unseen target languages. However, for different source languages, the quantity and difficulty vary greatly because of their different data scales and diverse phonological systems, which leads to task-quantity and task-difficulty imbalance issues and thus a failure of multilingual meta-learning ASR (MML-ASR). In this work, we solve this problem by developing a novel adversarial meta sampling (AMS) approach to improve MML-ASR. When sampling tasks in MML-ASR, AMS adaptively determines the task sampling probability for each source language. Specifically, for each source language, if the query loss is large, it means that its tasks are not well sampled to train ASR model in terms of its quantity and difficulty and thus should be sampled more frequently for extra learning. Inspired by this fact, we feed the historical task query loss of all source language domain into a network to learn a task sampling policy for adversarially increasing the current query loss of MML-ASR. Thus, the learnt task sampling policy can master the learning situation of each language and thus predicts good task sampling probability for each language for more effective learning. Finally, experiment results on two multilingual datasets show significant performance improvement when applying our AMS on MML-ASR, and also demonstrate the applicability of AMS to other low-resource speech tasks and transfer learning ASR approaches.

When does MAML Work the Best? An Empirical Study on Model-Agnostic Meta-Learning in NLP Applications

Model-Agnostic Meta-Learning for Natural Language Understanding Tasks in Finance

When MAML Can Adapt Fast and How to Assist When It Cannot

Alpha MAML: Adaptive Model-Agnostic Meta-Learning

How Does the Task Landscape Affect MAML Performance?

Cross-lingual Adaption Model-Agnostic Meta-Learning for Natural Language Understanding

On the Global Optimality of Model-Agnostic Meta-Learning

MAML-en-LLM: Model Agnostic Meta-Training of LLMs for Improved In-Context Learning

Generating Pseudo-labels Adaptively for Few-shot Model-Agnostic Meta-Learning

To Learn Effective Features: Understanding the Task-Specific Adaptation of MAML

How to train your MAML

MAML is a Noisy Contrastive Learner in Classification

ST-MAML: A Stochastic-Task based Method for Task-Heterogeneous Meta-Learning

Task Similarity Aware Meta Learning - Theory-Inspired Improvement on MAML.

Adversarial Meta Sampling for Multilingual Low-Resource Speech Recognition

Minimax and Neyman-Pearson Meta-Learning for Outlier Languages

BI-MAML: Balanced Incremental Approach for Meta Learning

Model-Agnostic Meta-Learning for Multilingual Hate Speech Detection

Rethinking the Number of Shots in Robust Model-Agnostic Meta-Learning

Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning

How to Train Your MAML to Excel in Few-Shot Classification