Abstract:Deep neural networks can yield good performance on various tasks but often require large amounts of data to train them. Meta-learning received considerable attention as one approach to improve the generalization of these networks from a limited amount of data. Whilst meta-learning techniques have been observed to be successful at this in various scenarios, recent results suggest that when evaluated on tasks from a different data distribution than the one used for training, a baseline that simply finetunes a pre-trained network may be more effective than more complicated meta-learning techniques such as MAML, which is one of the most popular meta-learning techniques. This is surprising as the learning behaviour of MAML mimics that of finetuning: both rely on re-using learned features. We investigate the observed performance differences between finetuning, MAML, and another meta-learning technique called Reptile, and show that MAML and Reptile specialize for fast adaptation in low-data regimes of similar data distribution as the one used for training. Our findings show that both the output layer and the noisy training conditions induced by data scarcity play important roles in facilitating this specialization for MAML. Lastly, we show that the pre-trained features as obtained by the finetuning baseline are more diverse and discriminative than those learned by MAML and Reptile. Due to this lack of diversity and distribution specialization, MAML and Reptile may fail to generalize to out-of-distribution tasks whereas finetuning can fall back on the diversity of the learned features.

Application of Meta Learning in Face Recognition

Review on the Application of Metalearning in Artificial Intelligence

Meta-learning approaches for learning-to-learn in deep learning: A survey

Learning Meta Pattern for Face Anti-Spoofing

Meta-learning-based Adversarial Training for Deep 3D Face Recognition on Point Clouds

A Comprehensive Overview and Survey of Recent Advances in Meta-Learning

Meta Transfer Learning for Facial Emotion Recognition

To Learn Effective Features: Understanding the Task-Specific Adaptation of MAML

A Task-Aware Attention-Based Method for Improved Meta-Learning.

A Joint Optimization Framework of the Embedding Model and Classifier for Meta-Learning

Meta-learning as a promising approach for few-shot cross-domain fault diagnosis: Algorithms, applications, and prospects

Meta-Learning in Neural Networks: A Survey

A study of Citespace-based deep learning applied to face recognition

Personalized Facial Beauty Assessment: a Meta-Learning Approach

Meta-Learning Empowered Meta-Face: Personalized Speaking Style Adaptation for Audio-Driven 3D Talking Face Animation

Advances and Challenges in Meta-Learning: A Technical Review

Deep Meta-Learning: Learning to Learn in the Concept Space

Faster Meta Update Strategy for Noise-Robust Deep Learning.

Understanding Transfer Learning and Gradient-Based Meta-Learning Techniques

Application of Face Recognition Method Under Deep Learning Algorithm in Embedded Systems

Facial Expression Recognition from a Single Face Image Based on Deep Learning and Broad Learning