Abstract:Meta-learning has demonstrated promising results in few-shot classification (FSC) by learning to solve new problems using prior knowledge. Bayesian methods are effective at characterizing uncertainty in FSC, which is crucial in high-risk fields. In this context, the logistic-softmax likelihood is often employed as an alternative to the softmax likelihood in multi-class Gaussian process classification due to its conditional conjugacy property. However, the theoretical property of logistic-softmax is not clear and previous research indicated that the inherent uncertainty of logistic-softmax leads to suboptimal performance. To mitigate these issues, we revisit and redesign the logistic-softmax likelihood, which enables control of the \textit{a priori} confidence level through a temperature parameter. Furthermore, we theoretically and empirically show that softmax can be viewed as a special case of logistic-softmax and logistic-softmax induces a larger family of data distribution than softmax. Utilizing modified logistic-softmax, we integrate the data augmentation technique into the deep kernel based Gaussian process meta-learning framework, and derive an analytical mean-field approximation for task-specific updates. Our approach yields well-calibrated uncertainty estimates and achieves comparable or superior results on standard benchmark datasets. Code is publicly available at \url{<a class="link-external link-https" href="https://github.com/keanson/revisit-logistic-softmax" rel="external noopener nofollow">this https URL</a>}.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is that in the existing few - shot classification (FSC), the theoretical properties of the logistic - softmax likelihood function are not clear and its inherent uncertainty leads to sub - optimal performance. Specifically: 1. **Limitations of the Logistic - softmax Likelihood Function**: - The logistic - softmax likelihood function is widely used in multi - class Gaussian process classification due to its conditional conjugate property, but its theoretical properties are still unclear. - Previous research has shown that the logistic - softmax likelihood function has inherent uncertainty, which will lead to poor performance in few - shot classification tasks. 2. **Improvement and Redesign of the Logistic - softmax Likelihood Function**: - To solve the above problems, the author re - examines and redesigns the logistic - softmax likelihood function, introducing a temperature parameter to control the prior confidence level. - By introducing the temperature parameter, the author can adjust the confidence of the logistic - softmax likelihood function more flexibly, thus improving its performance. 3. **Theoretical and Empirical Analysis**: - The author theoretically proves that softmax can be regarded as a special case of logistic - softmax, and logistic - softmax can induce a larger family of data distributions than softmax. - The experimental results show that the improved logistic - softmax achieves comparable or better results on standard benchmark datasets. 4. **Application in the Bayesian Meta - learning Framework**: - The author applies the improved logistic - softmax likelihood function to the Gaussian - process - based Bayesian meta - learning framework and derives an analytical mean - field approximation method for task - specific updates. - This method is more efficient than the existing Gibbs sampling and can achieve similar results in practice. In summary, this paper aims to solve the inherent uncertainty problem of the logistic - softmax likelihood function by redesigning it and apply it to the Bayesian meta - learning framework to improve the performance of few - shot classification tasks and the accuracy of uncertainty estimation.

Revisiting Logistic-softmax Likelihood in Bayesian Meta-Learning for Few-Shot Classification

Prototype Bayesian Meta-Learning for Few-Shot Image Classification

Amortized Bayesian Prototype Meta-learning: A New Probabilistic Meta-learning Approach to Few-shot Image Classification

A Hybrid Approach with Optimization-Based and Metric-Based Meta-Learner for Few-Shot Learning

Dual role of alpha-acetolactate decarboxylase in Lactococcus lactis subsp. lactis

Revisiting Unsupervised Meta-Learning via the Characteristics of Few-Shot Tasks

Improving metric-based few-shot learning with dynamically scaled softmax loss

Bayesian Evidential Learning for Few-Shot Classification

MALIBO: Meta-learning for Likelihood-free Bayesian Optimization

A metric-based meta-learning approach combined attention mechanism and ensemble learning for few-shot learning

Consistent Meta-Regularization for Better Meta-Knowledge in Few-Shot Learning

Gradient-Based Meta-Learning Using Uncertainty to Weigh Loss for Few-Shot Learning

Bridging the Gap Between Practice and PAC-Bayes Theory in Few-Shot Meta-Learning

Bayesian Model-Agnostic Meta-Learning

Few-Shot Learning With Class Imbalance

A Nested Bi-level Optimization Framework for Robust Few Shot Learning

Meta transfer evidence deep learning for trustworthy few-shot classification

One-Class Meta-Learning: Towards Generalizable Few-Shot Open-Set Classification

Scalable Multi-Modal Continual Meta-Learning

Few-shot Learning with Meta Metric Learners

A Stochastic Approach to Bi-Level Optimization for Hyperparameter Optimization and Meta Learning