Abstract:Self-supervised methods have shown remarkable progress in learning high-level semantics and low-level temporal correspondence. Building on these results, we take one step further and explore the possibility of integrating these two features to enhance object-centric representations. Our preliminary experiments indicate that query slot attention can extract different semantic components from the RGB feature map, while random sampling based slot attention can exploit temporal correspondence cues between frames to assist instance identification. Motivated by this, we propose a novel semantic-aware masked slot attention on top of the fused semantic features and correspondence maps. It comprises two slot attention stages with a set of shared learnable Gaussian distributions. In the first stage, we use the mean vectors as slot initialization to decompose potential semantics and generate semantic segmentation masks through iterative attention. In the second stage, for each semantics, we randomly sample slots from the corresponding Gaussian distribution and perform masked feature aggregation within the semantic area to exploit temporal correspondence patterns for instance identification. We adopt semantic- and instance-level temporal consistency as self-supervision to encourage temporally coherent object-centric representations. Our model effectively identifies multiple object instances with semantic structure, reaching promising results on unsupervised video object discovery. Furthermore, we achieve state-of-the-art performance on dense label propagation tasks, demonstrating the potential for object-centric analysis. The code is released at <a class="link-external link-https" href="https://github.com/shvdiwnkozbw/SMTC" rel="external noopener nofollow">this https URL</a>.

Invariant Slot Attention: Object Discovery with Slot-Centric Reference Frames

Adaptive Slot Attention: Object Discovery with Dynamic Slot Number

Spotlight Attention: Robust Object-Centric Learning With a Spatial Locality Prior

Shepherding Slots to Objects: Towards Stable and Robust Object-Centric Learning

Identifiable Object-Centric Representation Learning via Probabilistic Slot Attention

Improving Object-centric Learning with Query Optimization

SlotGNN: Unsupervised Discovery of Multi-Object Representations and Visual Dynamics

Semantics Meets Temporal Correspondence: Self-supervised Object-centric Learning in Videos

Cycle Consistency Driven Object Discovery

Attention Normalization Impacts Cardinality Generalization in Slot Attention

Masked Multi-Query Slot Attention for Unsupervised Object Discovery

Slot-VAE: Object-Centric Scene Generation with Slot Attention

Neural Slot Interpreters: Grounding Object Semantics in Emergent Slot Representations

Learning Global Object-Centric Representations via Disentangled Slot Attention

Enhancing Interpretable Object Abstraction via Clustering-based Slot Initialization

Slot Contrastive Networks: A Contrastive Approach for Representing Objects

Object-Centric Temporal Consistency via Conditional Autoregressive Inductive Biases

SlotLifter: Slot-guided Feature Lifting for Learning Object-centric Radiance Fields

Guided Latent Slot Diffusion for Object-Centric Learning

SPOT: Self-Training with Patch-Order Permutation for Object-Centric Learning with Autoregressive Transformers

Evolution of Siamese Visual Tracking with Slot Attention