Abstract:Weakly-Supervised Video Object Localization (WSVOL) involves localizing an object in videos using only video-level labels, also referred to as tags. State-of-the-art WSVOL methods like Temporal CAM (TCAM) rely on class activation mapping (CAM) and typically require a pre-trained CNN classifier. However, their localization accuracy is affected by their tendency to minimize the mutual information between different instances of a class and exploit temporal information during training for downstream tasks, e.g., detection and tracking. In the absence of bounding box annotation, it is challenging to exploit precise information about objects from temporal cues because the model struggles to locate objects over time. To address these issues, a novel method called transformer based CAM for videos (TrCAM-V), is proposed for WSVOL. It consists of a DeiT backbone with two heads for classification and localization. The classification head is trained using standard classification loss (CL), while the localization head is trained using pseudo-labels that are extracted using a pre-trained CLIP model. From these pseudo-labels, the high and low activation values are considered to be foreground and background regions, respectively. Our TrCAM-V method allows training a localization network by sampling pseudo-pixels on the fly from these regions. Additionally, a conditional random field (CRF) loss is employed to align the object boundaries with the foreground map. During inference, the model can process individual frames for real-time localization applications. Extensive experiments on challenging YouTube-Objects unconstrained video datasets show that our TrCAM-V method achieves new state-of-the-art performance in terms of classification and localization accuracy.

Task-Aware Weakly Supervised Object Localization With Transformer

LCTR: On Awakening the Local Continuity of Transformer for Weakly Supervised Object Localization

Adversarial Transformers for Weakly Supervised Object Localization

Weakly Supervised Object Localization Using Long-Range Semantic Foreground Activation

HiCT: Hierarchical Comprehend of Transformer for Weakly Supervised Object Localization

Semantic-Constraint Matching Transformer for Weakly Supervised Object Localization

Foreground Activation Maps for Weakly Supervised Object Localization.

Spatial-Aware Token for Weakly Supervised Object Localization

Leveraging Transformers for Weakly Supervised Object Localization in Unconstrained Videos

Weakly Supervised Object Localization As Domain Adaption.

Text to Point Cloud Localization with Relation-Enhanced Transformer.

Proxy Probing Decoder for Weakly Supervised Object Localization: A Baseline Investigation

Hierarchical Saliency Mapping for Weakly Supervised Object Localization Based on Class Activation Mapping

Reperceive Global Vision of Transformer for Remote Sensing Images Weakly Supervised Object Localization

Weakly supervised object localization via knowledge distillation based on foreground-background contrast

Re-Attention Transformer for Weakly Supervised Object Localization

EGSA: Enhanced and Global Semantic Activation for Weakly Supervised Object Localization.

Strengthen Learning Tolerance for Weakly Supervised Object Localization

LocLoc: Low-level Cues and Local-area Guides forWeakly Supervised Object Localization

LocLoc: Low-level Cues and Local-area Guides for Weakly Supervised Object Localization

CaFT: Clustering and Filter on Tokens of Transformer for Weakly Supervised Object Localization