Abstract:Domain generalization (DG) aims to learn a generic model from multiple observed source domains that generalizes well to arbitrary unseen target domains without further training. The major challenge in DG is that the model inevitably faces a severe overfitting issue due to the domain gap between source and target domains. To mitigate this problem, some dropout-based methods have been proposed to resist overfitting by discarding part of the representation of the intermediate layers. However, we observe that most of these methods only conduct the dropout operation in some specific layers, leading to an insufficient regularization effect on the model. We argue that applying dropout at multiple layers can produce stronger regularization effects, which could alleviate the overfitting problem on source domains more adequately than previous layer-specific dropout methods. In this paper, we develop a novel layer-wise and channel-wise dropout for DG, which randomly selects one layer and then randomly selects its channels to conduct dropout. Particularly, the proposed method can generate a variety of data variants to better deal with the overfitting issue. We also provide theoretical analysis for our dropout method and prove that it can effectively reduce the generalization error bound. Besides, we leverage the progressive scheme to increase the dropout ratio with the training progress, which can gradually boost the difficulty of training the model to enhance its robustness. Extensive experiments on three standard benchmark datasets have demonstrated that our method outperforms several state-of-the-art DG methods. Our code is available at <a class="link-external link-https" href="https://github.com/lingeringlight/PLACEdropout" rel="external noopener nofollow">this https URL</a>.

CLDM: Convolutional Layer Dropout Module.

Weighted Channel Dropout for Regularization of Deep Convolutional Neural Network

Correlation-based Structural Dropout for Convolutional Neural Networks

DCCD: Reducing Neural Network Redundancy Via Distillation

Wordreg: Mitigating the Gap Between Training and Inference with Worst-Case Drop Regularization

Effective and Efficient Dropout for Deep Convolutional Neural Networks

PLACE dropout: A Progressive Layer-wise and Channel-wise Dropout for Domain Generalization

Rethinking the Usage of Batch Normalization and Dropout in the Training of Deep Neural Networks

CamDrop

Attentiondrop For Convolutional Neural Networks

TargetDrop: A Targeted Regularization Method for Convolutional Neural Networks

FocusedDropout for Convolutional Neural Network

R-Drop: Regularized Dropout for Neural Networks.

Domain Generalization via Progressive Layer-wise and Channel-wise Dropout

R-Block: Regularized Block of Dropout for convolutional networks

LocalDrop: A Hybrid Regularization for Deep Neural Networks

A kernel-based weight decorrelation for regularizing CNNs

Beyond Dropout: Feature Map Distortion to Regularize Deep Neural Networks

ChannelDropBack: Forward-Consistent Stochastic Regularization for Deep Networks

On Dropping Clusters to Regularize Graph Convolutional Neural Networks