Abstract:Webly supervised learning has attracted increasing attention for its effectiveness in exploring publicly accessible data at scale without manual annotation. However, most existing methods of learning with web datasets are faced with challenges from label noise, and they have limited assumptions on clean samples under various noise. For instance, web images retrieved with queries of tiger cat (a cat species) and drumstick (a musical instrument) are almost dominated by images of tigers and chickens, which exacerbates the challenge of fine-grained visual concept learning. In this case, exploiting both web images and their associated texts is a requisite solution to combat real-world noise. In this paper, we propose Cross-modality Aligned Prototypes (CAPro), a unified prototypical contrastive learning framework to learn visual representations with correct semantics. For one thing, we leverage textual prototypes, which stem from the distinct concept definition of classes, to select clean images by text matching and thus disambiguate the formation of visual prototypes. For another, to handle missing and mismatched noisy texts, we resort to the visual feature space to complete and enhance individual texts and thereafter improve text matching. Such semantically aligned visual prototypes are further polished up with high-quality samples, and engaged in both cluster regularization and noise removal. Besides, we propose collective bootstrapping to encourage smoother and wiser label reference from appearance-similar instances in a manner of dictionary look-up. Extensive experiments on WebVision1k and NUS-WIDE (Web) demonstrate that CAPro well handles realistic noise under both single-label and multi-label scenarios. CAPro achieves new state-of-the-art performance and exhibits robustness to open-set recognition. Codes are available at <a class="link-external link-https" href="https://github.com/yuleiqin/capro" rel="external noopener nofollow">this https URL</a>.

Prototype Learning for Automatic Check-Out

Prototype-Based Supervised Contrastive Learning Method for Noisy Label Correction in Tire Defect Detection

Data Priming Network for Automatic Check-Out

Self-Supervised Multi-Category Counting Networks for Automatic Check-Out

Learning with Mixture of Prototypes for Out-of-Distribution Detection

RPC: A Large-Scale Retail Product Checkout Dataset

Online Prototype Learning for Online Continual Learning

PRO-CLIP: A CLIP-Based Category Measurement Network Through Prototype and Regularized Optimal Transportation

Empowering Retail Through Advanced Consumer Product Recognition Using Aquila Optimization Algorithm With Deep Learning

Prototype-based Classifier Learning for Long-Tailed Visual Recognition

Boosting Open-Domain Continual Learning via Leveraging Intra-domain Category-aware Prototype

An Enhanced Incremental Prototype Classifier using Subspace Representation Scheme

Learning Prototype Classifiers for Long-Tailed Recognition

APRICOT: Active Preference Learning and Constraint-Aware Task Planning with LLMs

Prototype-guided domain adaptive one-stage object detector for defect detection

Hunting Attributes: Context Prototype-Aware Learning for Weakly Supervised Semantic Segmentation

AOCIL: Exemplar-free Analytic Online Class Incremental Learning with Low Time and Resource Consumption

PCLDet: Prototypical Contrastive Learning for Fine-Grained Object Detection in Remote Sensing Images

Open-set long-tailed recognition via orthogonal prototype learning and false rejection correction

CAPro: Webly Supervised Learning with Cross-Modality Aligned Prototypes

Learning Few-Shot Object Placement with Intra-Category Transfer