Multi-Scale and Multi-Layer Contrastive Learning for Domain Generalization

Aristotelis Ballas,Christos Diou

DOI: https://doi.org/10.1109/TAI.2024.3377173

2024-05-10

Abstract:During the past decade, deep neural networks have led to fast-paced progress and significant achievements in computer vision problems, for both academia and industry. Yet despite their success, state-of-the-art image classification approaches fail to generalize well in previously unseen visual contexts, as required by many real-world applications. In this paper, we focus on this domain generalization (DG) problem and argue that the generalization ability of deep convolutional neural networks can be improved by taking advantage of multi-layer and multi-scaled representations of the network. We introduce a framework that aims at improving domain generalization of image classifiers by combining both low-level and high-level features at multiple scales, enabling the network to implicitly disentangle representations in its latent space and learn domain-invariant attributes of the depicted objects. Additionally, to further facilitate robust representation learning, we propose a novel objective function, inspired by contrastive learning, which aims at constraining the extracted representations to remain invariant under distribution shifts. We demonstrate the effectiveness of our method by evaluating on the domain generalization datasets of PACS, VLCS, Office-Home and NICO. Through extensive experimentation, we show that our model is able to surpass the performance of previous DG methods and consistently produce competitive and state-of-the-art results in all datasets

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper aims to address the problem of domain generalization in image classification for deep neural networks. Current state-of-the-art image classification methods perform poorly in unseen visual environments, while this cross-domain generalization ability is often required in practical applications. The paper proposes a multi-scale multi-layer contrastive learning framework that combines low-level and high-level features in deep neural networks, as well as representations of different scales, to enhance the decoupling of features in the network's latent space and learn domain-invariant image properties. In addition, the paper introduces a new loss function inspired by contrastive learning, to constrain the extracted representations to remain invariant under distributional changes. Through evaluations on generalization datasets in domains such as PACS, VLCS, Office-Home, and NICO, the method demonstrates performance surpassing previous domain generalization approaches and generates competitive state-of-the-art results on all datasets. The main contributions of the paper include proposing the M2 model structure, a novel contrastive learning loss function, and experimental validation on multiple domain generalization benchmarks.