Abstract:Domain adaptation (DA) attempts to transfer the knowledge from a labeled source domain to an unlabeled target domain that follows different distribution from the source. To achieve this, DA methods include a source classification objective L-S to extract the source knowledge and a domain alignment objective L-D to diminish the domain shift, ensuring knowledge transfer. Typically, former DA methods adopt some weight hyper-parameters to linearly combine the training objectives to form an overall objective L. However, the gradient directions of these objectives may conflict with each other due to domain shift. Under such circumstances, the linear optimization scheme might decrease the overall objective value at the expense of damaging one of the training objectives, leading to restricted solutions. In this paper, we rethink the optimization scheme for DA from a gradient-based perspective. We propose a Pareto Domain Adaptation (ParetoDA) approach to control the overall optimization direction, aiming to cooperatively optimize all training objectives. Specifically, to reach a desirable solution on the target domain, we design a surrogate loss mimicking target classification. To improve target-prediction accuracy to support the mimicking, we propose a target-prediction refining mechanism which exploits domain labels via Bayes' theorem. On the other hand, since prior knowledge of weighting schemes for objectives is often unavailable to guide optimization to approach the optimal solution on the target domain, we propose a dynamic preference mechanism to dynamically guide our cooperative optimization by the gradient of the surrogate loss on a held-out unlabeled target dataset. Our theoretical analyses show that the held-out data can guide but will not be over-fitted by the optimization. Extensive experiments on image classification and semantic segmentation benchmarks demonstrate the effectiveness of ParetoDA. Our code is available at https://github.com/BIT-DA/ParetoDA.

Resource Efficient Domain Adaptation

Dynamic Domain Adaptation for Efficient Inference

Selective Transfer with Reinforced Transfer Network for Partial Domain Adaptation.

Discriminative Label Consistent Domain Adaptation

Deep Discriminative Domain Adaptation.

Robust Data Geometric Structure Aligned Close yet Discriminative Domain Adaptation.

Multi-source Distilling Domain Adaptation

Maximal Domain Independent Representations Improve Transfer Learning

Domain-Invariant Feature Progressive Distillation with Adversarial Adaptive Augmentation for Low-Resource Cross-Domain NER.

Pareto Domain Adaptation

REACT: Remainder Adaptive Compensation for Domain Adaptive Object Detection

More is Better: Deep Domain Adaptation with Multiple Sources

Revisiting the Domain Shift and Sample Uncertainty in Multi-source Active Domain Transfer

Close Yet Distinctive Domain Adaptation.

MobileDA: Toward Edge-Domain Adaptation

Online Unsupervised Domain Adaptation Via Reducing Inter- and Intra-Domain Discrepancies

RobustDA: Lightweight Robust Domain Adaptation for Evolving Data at Edge

Domain Adaptation for Label Distribution Learning

Self-Adaptive Partial Domain Adaptation

Rethinking Distributional Matching Based Domain Adaptation

Multicomponent Adversarial Domain Adaptation: A General Framework.