Abstract:Video semantic segmentation is a pivotal aspect of video representation learning. However, significant domain shifts present a challenge in effectively learning invariant spatio-temporal features across the labeled source domain and unlabeled target domain for video semantic segmentation. To solve the challenge, we propose a novel DA-STC method for domain adaptive video semantic segmentation, which incorporates a bidirectional multi-level spatio-temporal fusion module and a category-aware spatio-temporal feature alignment module to facilitate consistent learning for domain-invariant features. Firstly, we perform bidirectional spatio-temporal fusion at the image sequence level and shallow feature level, leading to the construction of two fused intermediate video domains. This prompts the video semantic segmentation model to consistently learn spatio-temporal features of shared patch sequences which are influenced by domain-specific contexts, thereby mitigating the feature gap between the source and target domain. Secondly, we propose a category-aware feature alignment module to promote the consistency of spatio-temporal features, facilitating adaptation to the target domain. Specifically, we adaptively aggregate the domain-specific deep features of each category along spatio-temporal dimensions, which are further constrained to achieve cross-domain intra-class feature alignment and inter-class feature separation. Extensive experiments demonstrate the effectiveness of our method, which achieves state-of-the-art mIOUs on multiple challenging benchmarks. Furthermore, we extend the proposed DA-STC to the image domain, where it also exhibits superior performance for domain adaptive semantic segmentation. The source code and models will be made available at \url{https://github.com/ZHE-SAPI/DA-STC}.

Spatio-temporal Contrastive Domain Adaptation for Action Recognition

Spatio-Temporal Pixel-Level Contrastive Learning-based Source-Free Domain Adaptation for Video Semantic Segmentation

Cross-domain video action recognition via adaptive gradual learning

Simplifying Open-Set Video Domain Adaptation with Contrastive Learning

SADA: Semantic adversarial unsupervised domain adaptation for Temporal Action Localization

Annealing Temporal-Spatial Contrastive Learning for Multi-View Online Action Detection

Multi-Modal Domain Adaptation Across Video Scenes for Temporal Video Grounding

Channel-Temporal Attention for First-Person Video Domain Adaptation

Learning Discriminative Spatio-temporal Representations for Semi-supervised Action Recognition

Integrating multimodal contrastive learning with prototypical domain alignment for unsupervised domain adaptation of time series

Attentive spatial-temporal contrastive learning for self-supervised video representation

DA-STC: Domain Adaptive Video Semantic Segmentation via Spatio-Temporal Consistency.

Self-supervised pretext task collaborative multi-view contrastive learning for video action recognition

Video domain adaptation for semantic segmentation using perceptual consistency matching

Spatio-Temporal Adaptive Network with Bidirectional Temporal Difference for Action Recognition

Category Contrast for Unsupervised Domain Adaptation in Visual Tasks

Domain consensual contrastive learning for few-shot universal domain adaptation

CALDA: Improving Multi-Source Time Series Domain Adaptation with Contrastive Adversarial Learning

Fine-grained Temporal Contrastive Learning for Weakly-supervised Temporal Action Localization

Temporal Attentive Alignment for Large-Scale Video Domain Adaptation

Cross-domain Contrastive Learning for Unsupervised Domain Adaptation