Abstract:Data in the real world tends to exhibit a long-tailed label distribution, which poses great challenges for the training of neural networks in visual recognition. Existing methods tackle this problem mainly from the perspective of data quantity, i.e., the number of samples in each class. To be specific, they pay more attention to tail classes, like applying larger adjustments to the logit. However, in the training process, the quantity and difficulty of data are two intertwined and equally crucial problems. For some tail classes, the features of their instances are distinct and discriminative, which can also bring satisfactory accuracy; for some head classes, al-though with sufficient samples, the high semantic similarity with other classes and lack of discriminative features will bring bad accuracy. Based on these observations, we propose Adaptive Logit Adjustment Loss (ALA Loss) to ap-ply an adaptive adjusting term to the logit. The adaptive adjusting term is composed of two complementary factors: 1) quantity factor, which pays more attention to tail classes, and 2) difficulty factor, which adaptively pays more attention to hard instances in the training process. The difficulty factor can alleviate the over-optimization on tail yet easy in-stances and under-optimization on head yet hard instances. The synergy of the two factors can not only advance the performance on tail classes even further, but also promote the ac-curacy on head classes. Unlike previous logit adjusting methods that only concerned about data quantity, ALA Loss tackles the long-tailed problem from a more comprehensive, fine-grained and adaptive perspective. Extensive experimental re-sults show that our method achieves the state-of-the-art performance on challenging recognition benchmarks, including ImageNet-LT, iNaturalist 2018, and Places-LT.

Multiplicative Logit Adjustment Approximates Neural-Collapse-Aware Decision Boundary Adjustment

Adjusting Logit in Gaussian Form for Long-Tailed Visual Recognition

Adaptive Logit Adjustment Loss for Long-Tailed Visual Recognition

The Exploration of Neural Collapse under Imbalanced Data

Long-tailed Visual Recognition via Gaussian Clouded Logit Adjustment

Neural Collapse in Multi-label Learning with Pick-all-label Loss

Myopic Loss Aversion or Equate-to-differentiate Heuristic? A Heuristic Decision Making Model for Both Single and Aggregated Plays.

Local and Global Logit Adjustments for Long-Tailed Learning

Gradient-Aware Logit Adjustment Loss for Long-tailed Classifier

A novel M-Lognormal–Burr regression model with varying threshold for modeling heavy-tailed claim severity data

A Complete Decomposition of KL Error using Refined Information and Mode Interaction Selection

Dual-Head Knowledge Distillation: Enhancing Logits Utilization with an Auxiliary Head

Online Continual Learning via Logit Adjusted Softmax

Rethinking Classifier Re-Training in Long-Tailed Recognition: A Simple Logits Retargeting Approach

Improving Long-Tailed Classification from Instance Level

Inducing Neural Collapse in Imbalanced Learning: Do We Really Need a Learnable Classifier at the End of Deep Neural Network?

Taming heavy-tailed features by shrinkage

Avoiding spurious correlations via logit correction

Improving Logits-based Detector without Logits from Black-box LLMs

Provably Efficient Reinforcement Learning with Multinomial Logit Function Approximation

A Neural Implementation of Wald’s Sequential Probability Ratio Test