Abstract:Domain Generalized Semantic Segmentation (DGSS) seeks to utilize source domain data exclusively to enhance the generalization of semantic segmentation across unknown target domains. Prevailing studies predominantly concentrate on feature normalization and domain randomization, these approaches exhibit significant limitations. Feature normalization-based methods tend to confuse semantic features in the process of constraining the feature space distribution, resulting in classification misjudgment. Domain randomization-based methods frequently incorporate domain-irrelevant noise due to the uncontrollability of style transformations, resulting in segmentation ambiguity. To address these challenges, we introduce a novel framework, named SCSD for Semantic Consistency prediction and Style Diversity generalization. It comprises three pivotal components: Firstly, a Semantic Query Booster is designed to enhance the semantic awareness and discrimination capabilities of object queries in the mask decoder, enabling cross-domain semantic consistency prediction. Secondly, we develop a Text-Driven Style Transform module that utilizes domain difference text embeddings to controllably guide the style transformation of image features, thereby increasing inter-domain style diversity. Lastly, to prevent the collapse of similar domain feature spaces, we introduce a Style Synergy Optimization mechanism that fortifies the separation of inter-domain features and the aggregation of intra-domain features by synergistically weighting style contrastive loss and style aggregation loss. Extensive experiments demonstrate that the proposed SCSD significantly outperforms existing state-of-theart methods. Notably, SCSD trained on GTAV achieved an average of 49.11 mIoU on the four unseen domain datasets, surpassing the previous state-of-the-art method by +4.08 mIoU. Code is available at <a class="link-external link-https" href="https://github.com/nhw649/SCSD" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve two key challenges in **Domain Generalized Semantic Segmentation (DGSS)**: 1. **The semantic confusion problem of feature normalization methods**: - Existing feature - normalization - based methods are prone to confusing the semantic features of different classes when constraining the feature space distribution, leading to misclassification. - Formula representation: Suppose \(F\) represents the feature space, and \(C_i\) and \(C_j\) represent different classes. Feature normalization methods may make \(F(C_i)\approx F(C_j)\), thus leading to classification errors. 2. **The noise interference problem of domain randomization methods**: - Methods based on domain randomization often introduce noise unrelated to the domain due to the uncontrollability of style transformation, resulting in blurry segmentation. - Formula representation: Let \(I_s\) be the source - domain image and \(T_d\) be the target - domain style transformation. If \(T_d(I_s)\) introduces noise \(N\), then \(T_d(I_s)=I_t + N\), where \(I_t\) is the ideal target - domain image. To address these challenges, the authors propose a new framework named **SCSD (Semantic Consistency and Style Diversity)**, which specifically includes the following three main components: - **Semantic Query Booster (SQB)**: - Designed to enhance the semantic perception and discrimination ability of object queries in the mask decoder, achieving cross - domain semantic consistency prediction. - Key formula: By calculating the semantic similarity map \(S = F_v\cdot E_t^T\), where \(F_v\) is the dense visual feature and \(E_t\) is the text embedding, thereby enhancing semantic association. - **Text - Driven Style Transform (TDST)**: - Use domain - difference text embeddings to controllably guide the style transformation of image features, increasing cross - domain style diversity. - Key formula: Generate style - difference features \(F_d\) through domain - difference embeddings \(E_d = E_s - E_g\), where \(E_s\) and \(E_g\) are text embeddings of specific domains and general domains respectively. - **Style Synergy Optimization (SSO)**: - Strengthen cross - domain feature separation and intra - domain feature aggregation by synergistically weighting the style contrast loss \(L_{sc}\) and the style aggregation loss \(L_{sa}\). - Key formula: The overall loss function \(L_s=w_{sa}\cdot L_{sc}+w_{sc}\cdot L_{sa}\), where the weights \(w_{sa}\) and \(w_{sc}\) are adaptively adjusted according to the loss change trend. Through these innovative components, SCSD significantly improves the generalization ability of the model in unknown target domains, especially outperforming existing methods in terms of the mean intersection - over - union (mIoU) on multiple unseen datasets.

Exploring Semantic Consistency and Style Diversity for Domain Generalized Semantic Segmentation

IS2Net: Intra-domain Semantic and Inter-domain Style Enhancement for Semi-supervised Medical Domain Generalization

When Masked Image Modeling Meets Source-free Unsupervised Domain Adaptation: Dual-Level Masked Network for Semantic Segmentation

Semantic-Aware Domain Generalized Segmentation.

SCDA: A Style and Content Domain Adaptive Semantic Segmentation Method for Remote Sensing Images

Style Projected Clustering for Domain Generalized Semantic Segmentation

DGInStyle: Domain-Generalizable Semantic Segmentation with Image Diffusion Models and Stylized Semantic Control

CDDSA: Contrastive domain disentanglement and style augmentation for generalizable medical image segmentation

Sequential Style Consistency Learning for Domain-Generalizable Text Recognition.

Shape Robustness in Style Enhanced Cross Domain Semantic Segmentation

Inter-Class and Inter-Domain Semantic Augmentation for Domain Generalization

Learning deep semantic segmentation network under multiple weakly-supervised constraints for cross-domain remote sensing image semantic segmentation

Single Domain Generalization Method for Remote Sensing Image Segmentation via Category Consistency on Domain Randomization

Pixel-Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation

Unsupervised Domain Adaptive Point Cloud Semantic Segmentation.

Style Adaptation for Domain-adaptive Semantic Segmentation

Uncertainty-Aware Consistency Regularization for Cross-Domain Semantic Segmentation

Intra- & Extra-Source Exemplar-Based Style Synthesis for Improved Domain Generalization

Devil is in Channels: Contrastive Single Domain Generalization for Medical Image Segmentation

S&GDA: An Unsupervised Domain Adaptive Semantic Segmentation Framework Considering Both Imaging Scene and Geometric Domain Shifts

Generalizable model-agnostic semantic segmentation via target-specific normalization