Abstract:Webly supervised learning has attracted increasing attention for its effectiveness in exploring publicly accessible data at scale without manual annotation. However, most existing methods of learning with web datasets are faced with challenges from label noise, and they have limited assumptions on clean samples under various noise. For instance, web images retrieved with queries of tiger cat (a cat species) and drumstick (a musical instrument) are almost dominated by images of tigers and chickens, which exacerbates the challenge of fine-grained visual concept learning. In this case, exploiting both web images and their associated texts is a requisite solution to combat real-world noise. In this paper, we propose Cross-modality Aligned Prototypes (CAPro), a unified prototypical contrastive learning framework to learn visual representations with correct semantics. For one thing, we leverage textual prototypes, which stem from the distinct concept definition of classes, to select clean images by text matching and thus disambiguate the formation of visual prototypes. For another, to handle missing and mismatched noisy texts, we resort to the visual feature space to complete and enhance individual texts and thereafter improve text matching. Such semantically aligned visual prototypes are further polished up with high-quality samples, and engaged in both cluster regularization and noise removal. Besides, we propose collective bootstrapping to encourage smoother and wiser label reference from appearance-similar instances in a manner of dictionary look-up. Extensive experiments on WebVision1k and NUS-WIDE (Web) demonstrate that CAPro well handles realistic noise under both single-label and multi-label scenarios. CAPro achieves new state-of-the-art performance and exhibits robustness to open-set recognition. Codes are available at <a class="link-external link-https" href="https://github.com/yuleiqin/capro" rel="external noopener nofollow">this https URL</a>.

Noise-Tolerant Hybrid Prototypical Learning with Noisy Web Data

Rethinking Noisy Label Learning in Real-world Annotation Scenarios from the Noise-type Perspective

Prototype-Based Supervised Contrastive Learning Method for Noisy Label Correction in Tire Defect Detection

Noise is the Fatal Poison: A Noise-aware Network for Noisy Dataset Classification

Label Correction Using Contrastive Prototypical Classifier for Noisy Label Learning.

ProtoNet: Learning from Web Data with Memory

Learning from Open-Set Noisy Labels Based on Multi-Prototype Modeling

CAPro: Webly Supervised Learning with Cross-Modality Aligned Prototypes

Prototypical Classifier for Robust Class-Imbalanced Learning

MoPro: Webly Supervised Learning with Momentum Prototypes

Online Prototype Learning for Online Continual Learning

Extracting Clean and Balanced Subset for Noisy Long-tailed Classification

Class2Simi: A Noise Reduction Perspective on Learning with Noisy Labels

Hybrid Attention-Based Prototypical Networks for Noisy Few-Shot Relation Classification.

Robust Image Classification with Noisy Labels by Negative Learning and Feature Space Renormalization

Hybrid Noise-Oriented Multilabel Learning.

An joint end-to-end framework for learning with noisy labels

Prototypical Mixing and Retrieval-Based Refinement for Label Noise-Resistant Image Retrieval

Learning with Noisy Foundation Models

Multi-Class Classification from Noisy-Similarity-Labeled Data