Abstract:Modern deep neural networks are prone to learn domain-dependent shortcuts and thus usually suffer from severe performance degradation when tested in unseen target domains due to their poor ability of out-of-distribution generalization, which significantly limits the real-world applications. The main reason is the domain shift lying in the large distribution gap between source and unseen target data. To this end, this paper takes a step towards training robust models for domain generalizable visual tasks, which mainly focuses on learning domain-invariant visual representation to alleviate the domain shift. Specifically, we first propose an effective Hierarchical Visual Transformation (HVT) network to (1) first transform the training sample hierarchically into new domains with diverse distributions from three levels: Global, Local, and Pixel, (2) then maximize the visual discrepancy between the source domain and new domains, and minimize the cross-domain feature inconsistency to capture domain-invariant features. Besides, we further enhance the HVT network by introducing the environment-invariant learning. To be specific, we enforce the invariance of the visual representation across automatically inferred environments by minimizing invariant learning loss that considers the weighted average of environmental losses. In this way, we can prevent the model from relying on the spurious features for prediction, thus helping the model to effectively learn domain-invariant representation and narrow the domain gap in various visual matching and recognition tasks, such as stereo matching, pedestrian retrieval, and image classification. We term our extended HVT as EHVT to show distinction. We integrate our EHVT network into different models and evaluate its effectiveness and compatibility on several public benchmark datasets. Extensive experiments clearly show that our EHVT can substantially enhance the generalization performance in various tasks. Our codes are available at https://github.com/cty8998/EHVT-VisualDG.

Improving Visual Recognition with Hyperbolical Visual Hierarchy Mapping

Long-Term Map-Based Visual Localization: Analysis of Individual Components of a Hierarchical Pipeline

Hierarchical Gate Network for Fine-Grained Visual Recognition.

Learning Visual Hierarchies with Hyperbolic Embeddings

Learning Hierarchical Visual Transformation for Domain Generalizable Visual Matching and Recognition

Learning Hierarchical Image Segmentation For Recognition and By Recognition

Learning Hierarchical Dynamics with Spatial Adjacency for Image Enhancement

HiT: Building Mapping With Hierarchical Transformers

Embedding Visual Hierarchy with Deep Networks for Large-Scale Visual Recognition

Reassessing Hierarchical Representation for Action Recognition in Still Images

Dynamic Hierarchical Mimicking Towards Consistent Optimization Objectives

HiH: A Multi-modal Hierarchy in Hierarchy Network for Unconstrained Gait Recognition

HVT: A Comprehensive Vision Framework for Learning in Non-Euclidean Space

Hierarchical visual-semantic interaction for scene text recognition

Hierarchical Dynamic Masks for Visual Explanation of Neural Networks

Why are hyperbolic neural networks effective? A study on hierarchical representation capability

Deep Hierarchical Semantic Segmentation

HAFormer: Unleashing the Power of Hierarchy-Aware Features for Lightweight Semantic Segmentation

Fully Hyperbolic Convolutional Neural Networks for Computer Vision

Learning 3D Semantics from Pose-Noisy 2D Images with Hierarchical Full Attention Network

Exploring Hierarchical Information in Hyperbolic Space for Self-Supervised Image Hashing