Abstract:Few-shot learning or meta-learning leverages the data scarcity problem in machine learning. Traditionally, training data requires a multitude of samples and labeling for supervised learning. To address this issue, we propose a one-shot unsupervised meta-learning to learn the latent representation of the training samples. We use augmented samples as the query set during the training phase of the unsupervised meta-learning. A temperature-scaled cross-entropy loss is used in the inner loop of meta-learning to prevent overfitting during unsupervised learning. The learned parameters from this step are applied to the targeted supervised meta-learning in a transfer-learning fashion for initialization and fast adaptation with improved accuracy. The proposed method is model agnostic and can aid any meta-learning model to improve accuracy. We use model agnostic meta-learning (MAML) and relation network (RN) on Omniglot and mini-Imagenet datasets to demonstrate the performance of the proposed method. Furthermore, a meta-learning model with the proposed initialization can achieve satisfactory accuracy with significantly fewer training samples.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to use unsupervised representation learning to assist semi - supervised meta - learning (meta - learning) in the case of a small number of samples, so as to improve the accuracy and generalization ability of the model. Traditional machine learning methods require a large amount of labeled data for supervised learning, and when the test domain deviates slightly from the training domain, the performance of these algorithms will be affected. In addition, when introducing new categories, it is usually necessary to train the model from scratch. To solve these problems, the author proposes a new method, that is, using unsupervised meta - learning to learn the latent representation of training samples and using augmented samples as the query set. Specifically, the main contributions of the paper include: 1. **Proposing a more effective data augmentation technique**: Combining SimCLR and other methods to generate the query set, thereby improving the performance of the model. 2. **Using temperature - scaled SoftMax in the inner loop of MAML**: Reducing the over - fitting problem in the unsupervised training process. 3. **Replacing random initialization with unsupervised representation learning**: Through intrinsic feature learning, without the need for a large amount of data labeling, and then performing supervised meta - learning to improve accuracy. 4. **Proving the model - independence of the two - step meta - learning method**: It can be applied to any existing meta - learning model and still performs well on partially labeled data. ### Method Overview #### Step 1: Unsupervised Learning - **Data Preparation**: Extract samples from the whole or part of the data set without using any label information. - **Support Set and Query Set Generation**: Randomly extract samples and apply data augmentation to generate the query set. - **Classifier Training**: Use the temperature - scaled SoftMax activation function (only in the inner loop of MAML) to reduce over - fitting. #### Step 2: Semi - supervised Meta - learning - **Parameter Transfer**: Use the parameters learned in the unsupervised learning stage to initialize supervised meta - learning. - **Fine - tuning and Improvement**: Perform fine - tuning through the regular meta - learning steps to improve accuracy. ### Experimental Results The paper has been verified on two benchmark data sets, Omniglot and mini - Imagenet. The experimental results show that the proposed method performs well on different data augmentation techniques and classifier architectures (such as MAML and RN), especially in 5 - way 1 - shot and 20 - way 1 - shot classification tasks. ### Summary This paper proposes an innovative unsupervised representation learning method, which combines meta - learning and data augmentation techniques, aiming to solve the data scarcity problem in few - sample learning and significantly improve the accuracy and generalization ability of the model.

Unsupervised Representation Learning to Aid Semi-Supervised Meta Learning

Unsupervised meta-learning for few-shot learning

Revisiting Unsupervised Meta-Learning via the Characteristics of Few-Shot Tasks

Representation Based and Attention Augmented Meta Learning

Meta-Learning for Semi-Supervised Few-Shot Classification

Semi-Supervised Meta-Learning Via Self-Training

A metric-based meta-learning approach combined attention mechanism and ensemble learning for few-shot learning

Meta-free few-shot learning via representation learning with weight averaging

Self-supervised pairwise-sample resistance model for few-shot classification

Meta-learning methodology based on meta-unsupervised algorithm for meta-model selection to solve few-shot base-tasks

Unsupervised Meta-Learning via Dynamic Head and Heterogeneous Task Construction for Few-Shot Classification

Meta-Tasks: An alternative view on Meta-Learning Regularization

Pseudo-Labeling Based Practical Semi-Supervised Meta-Training for Few-Shot Learning

MetaConcept: Learn to Abstract Via Concept Graph for Weakly-Supervised Few-Shot Learning

Meta Learning to Bridge Vision and Language Models for Multimodal Few-Shot Learning

Meta Omnium: A Benchmark for General-Purpose Learning-To-Learn

Meta-SGD: Learning to Learn Quickly for Few-Shot Learning

Meta-learning autoencoders for few-shot prediction

Boosting Meta-Training with Base Class Information for Few-Shot Learning