Abstract:Real world data often exhibits a long-tailed and open-ended (i.e., with unseen classes) distribution. A practical recognition system must balance between majority (head) and minority (tail) classes, generalize across the distribution, and acknowledge novelty upon the instances of unseen classes (open classes). We define Open Long-Tailed Recognition++ (OLTR++) as learning from such naturally distributed data and optimizing for the classification accuracy over a balanced test set which includes both known and open classes. OLTR++ handles imbalanced classification, few-shot learning, open-set recognition, and active learning in one integrated algorithm, whereas existing classification approaches often focus only on one or two aspects and deliver poorly over the entire spectrum. The key challenges are: 1) how to share visual knowledge between head and tail classes, 2) how to reduce confusion between tail and open classes, and 3) how to actively explore open classes with learned knowledge. Our algorithm, OLTR++, maps images to a feature space such that visual concepts can relate to each other through a memory association mechanism and a learned metric (dynamic meta-embedding) that both respects the closed world classification of seen classes and acknowledges the novelty of open classes. Additionally, we propose an active learning scheme based on visual memory, which learns to recognize open classes in a data-efficient manner for future expansions. On three large-scale open long-tailed datasets we curated from ImageNet (object-centric), Places (scene-centric), and MS1M (face-centric) data, as well as three standard benchmarks (CIFAR-10-LT, CIFAR-100-LT, and iNaturalist-18), our approach, as a unified framework, consistently demonstrates competitive performance. Notably, our approach also shows strong potential for the active exploration of open classes and the fairness analysis of minority groups.

Open-set learning context recognizing in mobile learning: Problem and methodology

Convolutional Prototype Network for Open Set Recognition

Open-Set Recognition of Screen Defects with Negative-Guided Augmented Prototype Generator and Open Feature Generation

Know Yourself Better: Diverse Discriminative Feature Learning Improves Open Set Recognition

Dynamic Against Dynamic: An Open-set Self-learning Framework

Contrastive Open Set Recognition

Adversarial Reciprocal Points Learning for Open Set Recognition

Open-set long-tailed recognition via orthogonal prototype learning and false rejection correction

LMC: Large Model Collaboration with Cross-assessment for Training-Free Open-Set Object Recognition

Open-Set Object Detection Using Classification-free Object Proposal and Instance-level Contrastive Learning

Learning Placeholders for Open-Set Recognition

Open Long-Tailed Recognition In A Dynamic World

Online Learning with Mobile Sensor Data for User Recognition

P-ODN: Prototype based Open Deep Network for Open Set Recognition

Open-environment Machine Learning.

OpenSlot: Mixed Open-set Recognition with Object-centric Learning

Deep Active Learning in the Open World

Online Context Learning for Socially-compliant Navigation

Towards open-set text recognition via label-to-prototype learning

Learning Open Set Network with Discriminative Reciprocal Points

Learning Discriminative Feature Representation for Open Set Action Recognition