Abstract:Real world data often exhibits a long-tailed and open-ended (i.e., with unseen classes) distribution. A practical recognition system must balance between majority (head) and minority (tail) classes, generalize across the distribution, and acknowledge novelty upon the instances of unseen classes (open classes). We define Open Long-Tailed Recognition++ (OLTR++) as learning from such naturally distributed data and optimizing for the classification accuracy over a balanced test set which includes both known and open classes. OLTR++ handles imbalanced classification, few-shot learning, open-set recognition, and active learning in one integrated algorithm, whereas existing classification approaches often focus only on one or two aspects and deliver poorly over the entire spectrum. The key challenges are: 1) how to share visual knowledge between head and tail classes, 2) how to reduce confusion between tail and open classes, and 3) how to actively explore open classes with learned knowledge. Our algorithm, OLTR++, maps images to a feature space such that visual concepts can relate to each other through a memory association mechanism and a learned metric (dynamic meta-embedding) that both respects the closed world classification of seen classes and acknowledges the novelty of open classes. Additionally, we propose an active learning scheme based on visual memory, which learns to recognize open classes in a data-efficient manner for future expansions. On three large-scale open long-tailed datasets we curated from ImageNet (object-centric), Places (scene-centric), and MS1M (face-centric) data, as well as three standard benchmarks (CIFAR-10-LT, CIFAR-100-LT, and iNaturalist-18), our approach, as a unified framework, consistently demonstrates competitive performance. Notably, our approach also shows strong potential for the active exploration of open classes and the fairness analysis of minority groups.

Learning Discriminative Feature Representation for Open Set Action Recognition

Convolutional Prototype Network for Open Set Recognition

Enlarging Instance-specific and Class-specific Information for Open-set Action Recognition

Adversarial Reciprocal Points Learning for Open Set Recognition

Know Yourself Better: Diverse Discriminative Feature Learning Improves Open Set Recognition

Class-Specific Semantic Reconstruction for Open Set Recognition

Exploring Reinforced Class Separability and Discriminative Representations for SAR Target Open Set Recognition

Contrastive Open Set Recognition

A SAR Open-Set Recognition Method Aided by Hierarchically Reconstructive Latent Representation Learning

Navigating Open Set Scenarios for Skeleton-based Action Recognition

Exploring Diverse Representations for Open Set Recognition

ODN: Opening the Deep Network for Open-Set Action Recognition.

Synergetic proto-pull and reciprocal points for open set recognition

Learning Discriminative Representations for Skeleton Based Action Recognition

Open Set Audio Classification Using Autoencoders Trained on Few Data

An Entropy-Awareness Meta-Learning Method for SAR Open-Set ATR

Class-aware Variational Auto-encoder for Open Set Recognition

SOAR: Scene-debiasing Open-set Action Recognition

Logit prototype learning with active multimodal representation for robust open-set recognition

Open Long-Tailed Recognition In A Dynamic World

Evidential Deep Learning for Open Set Action Recognition