Abstract:Pattern recognition is significantly challenging in real-world scenarios by the variability of visual statistics. Therefore, most existing algorithms relying on the independent identically distributed assumption of training and test data suffer from the poor generalization capability of inference on unseen testing datasets. Although numerous studies, including domain discriminator or domain-invariant feature learning, are proposed to alleviate this problem, the data-driven property and lack of interpretation of their principle throw researchers and developers off. Consequently, this dilemma incurs us to rethink the essence of networks' generalization. An observation that visual patterns cannot be discriminative after style transfer inspires us to take careful consideration of the importance of style features and content features. Does the style information related to the domain bias? How to effectively disentangle content and style features across domains? In this article, we first investigate the effect of feature normalization on domain adaptation. Based on it, we propose a novel normalization module to adaptively leverage the propagated information through each channel and batch of features called disentangling batch instance normalization (D-BIN). In this module, we explicitly explore domain-specific and domaininvariant feature disentanglement. We maneuver contrastive learning to encourage images with the same semantics from different domains to have similar content representations while having dissimilar style representations. Furthermore, we construct both self-form and dual-form regularizers for preserving the mutual information (MI) between feature representations of the normalization layer in order to compensate for the loss of discriminative information and effectively match the distributions across domains. D-BIN and the constrained term can be simply plugged into state-of-the-art (SOTA) networks to improve their performance. In the end, experiments, including domain adaptation and generalization, conducted on different datasets have proven their effectiveness.

Subdomain contraction in deep networks for robust representation learning

Deep Co-Attention Network for Multi-View Subspace Learning

Improving the Robustness of Deep Convolutional Neural Networks Through Feature Learning

DomainDrop: Suppressing Domain-Sensitive Channels for Domain Generalization

Regularizing Deep Convolutional Neural Networks with a Structured Decorrelation Constraint.

Denoising in Representation Space Via Data-Dependent Regularization for Better Representation

DSDRNet: Disentangling Representation and Reconstruct Network for Domain Generalization

Subclass-Dominant Label Noise: A Counterexample for the Success of Early Stopping

Feature Equilibrium: An Adversarial Training Method to Improve Representation Learning

Mask-Guided Divergence Loss Improves the Generalization and Robustness of Deep Neural Network

Robust Subspace Clustering Network with Dual-Domain Regularization

Learning Representative Features Via Constrictive Annular Loss for Image Classification

ACE: A Coarse-to-Fine Learning Framework for Reliable Representation Learning Against Label Noise

Learning Structured and Non-Redundant Representations with Deep Neural Networks

Learning Compact Representations of Neural Networks using DiscriminAtive Masking (DAM)

Semi-supervised Domain Adaptation Via Subspace Exploration

Domain Adaptation by Stacked Local Constraint Auto-Encoder Learning

Robust Learning with Progressive Data Expansion Against Spurious Correlation

Deep Residual Correction Network for Partial Domain Adaptation

D-BIN: A Generalized Disentangling Batch Instance Normalization for Domain Adaptation.

Attention Diversification for Domain Generalization