Abstract:Self-supervised methods have shown remarkable progress in learning high-level semantics and low-level temporal correspondence. Building on these results, we take one step further and explore the possibility of integrating these two features to enhance object-centric representations. Our preliminary experiments indicate that query slot attention can extract different semantic components from the RGB feature map, while random sampling based slot attention can exploit temporal correspondence cues between frames to assist instance identification. Motivated by this, we propose a novel semantic-aware masked slot attention on top of the fused semantic features and correspondence maps. It comprises two slot attention stages with a set of shared learnable Gaussian distributions. In the first stage, we use the mean vectors as slot initialization to decompose potential semantics and generate semantic segmentation masks through iterative attention. In the second stage, for each semantics, we randomly sample slots from the corresponding Gaussian distribution and perform masked feature aggregation within the semantic area to exploit temporal correspondence patterns for instance identification. We adopt semantic- and instance-level temporal consistency as self-supervision to encourage temporally coherent object-centric representations. Our model effectively identifies multiple object instances with semantic structure, reaching promising results on unsupervised video object discovery. Furthermore, we achieve state-of-the-art performance on dense label propagation tasks, demonstrating the potential for object-centric analysis. The code is released at <a class="link-external link-https" href="https://github.com/shvdiwnkozbw/SMTC" rel="external noopener nofollow">this https URL</a>.

Identifiable Object-Centric Representation Learning via Probabilistic Slot Attention

Invariant Slot Attention: Object Discovery with Slot-Centric Reference Frames

Spotlight Attention: Robust Object-Centric Learning With a Spatial Locality Prior

Object-Centric Learning with Slot Mixture Module

Explicitly Disentangled Representations in Object-Centric Learning

Learning Global Object-Centric Representations via Disentangled Slot Attention

Adaptive Slot Attention: Object Discovery with Dynamic Slot Number

Object-centric architectures enable efficient causal representation learning

Sensitivity of Slot-Based Object-Centric Models to their Number of Slots

Object Pursuit: Building a Space of Objects Via Discriminative Weight Generation

Leveraging Image Augmentation for Object Manipulation: Towards Interpretable Controllability in Object-Centric Learning

Shepherding Slots to Objects: Towards Stable and Robust Object-Centric Learning

Attention Normalization Impacts Cardinality Generalization in Slot Attention

Semantics Meets Temporal Correspondence: Self-supervised Object-centric Learning in Videos

Provable Compositional Generalization for Object-Centric Learning

SlotGNN: Unsupervised Discovery of Multi-Object Representations and Visual Dynamics

Improving Object-centric Learning with Query Optimization

Guided Latent Slot Diffusion for Object-Centric Learning

Masked Multi-Query Slot Attention for Unsupervised Object Discovery

Simplified priors for Object-Centric Learning

Cycle Consistency Driven Object Discovery