Abstract:Conventional Multi-Source Free Domain Adaptation (MSFDA) assumes that each source domain provides a single source model, and all source models adopt a uniform architecture. This paper introduces Zoo-MSFDA, a more general setting that allows each source domain to offer a zoo of multiple source models with different architectures. While it enriches the source knowledge, Zoo-MSFDA risks being dominated by suboptimal/harmful models. To address this issue, we theoretically analyze the model selection problem in Zoo-MSFDA, and introduce two principles: transferability principle and diversity principle. Recognizing the challenge of measuring transferability, we subsequently propose a novel Source-Free Unsupervised Transferability Estimation (SUTE). It enables assessing and comparing transferability across multiple source models with different architectures under domain shift, without requiring target labels and source data. Based on above, we introduce a Selection, Ensemble, and Adaptation (SEA) framework to address Zoo-MSFDA, which consists of: 1) source models selection based on the proposed principles and SUTE; 2) ensemble construction based on SUTE-estimated transferability; 3) target-domain adaptation of the ensemble model. Evaluations demonstrate that our SEA framework, with the introduced Zoo-MSFDA setting, significantly improves adaptation performance (e.g., 13.5% on DomainNet). Additionally, our SUTE achieves state-of-the-art performance in transferability estimation.
What problem does this paper attempt to address?
### What problems does this paper attempt to solve?
This paper aims to solve several key problems in multi - source unsupervised domain adaptation (Multi - Source - Free Domain Adaptation, MSFDA). Specifically, it proposes a new MSFDA setting, called Zoo - MSFDA, and develops corresponding frameworks and methods for this setting. The following are the main objectives of the paper:
1. **Expand the utilization of multi - source models**:
- Traditional MSFDA assumes that each source domain only provides one source model with a unified architecture. However, source models with different architectures can bring more abundant knowledge. Therefore, the paper introduces the Zoo - MSFDA setting, allowing each source domain to provide multiple source models with different architectures, forming a "model zoo".
2. **Select appropriate source models**:
- In Zoo - MSFDA, since there are a large number of source models with different architectures, how to select appropriate models becomes a challenge. Through theoretical analysis, the paper proposes two selection principles: the transferability principle and the diversity principle. The transferability principle emphasizes selecting models that can accurately capture the data distribution of the target domain; the diversity principle emphasizes ensuring the diversity and complementarity among the selected models to enhance collective knowledge.
3. **Evaluate transferability**:
- In order to evaluate the transferability of source models with different architectures without accessing target labels and source data, the paper proposes a new unsupervised transferability evaluation method (Source - Free Unsupervised Transferability Estimation, SUTE). SUTE indirectly evaluates the transfer ability of models through three key indicators (individual certainty, semantic consistency, global dispersion).
4. **Construct an adaptation framework**:
- The paper proposes a new framework Selection, Ensemble, and Adaptation (SEA) for handling Zoo - MSFDA. This framework includes three steps:
1. **Source model selection**: Based on the proposed transferability principle and diversity principle, select appropriate source models.
2. **Model integration**: Build an integrated model according to the transferability evaluated by SUTE.
3. **Target domain adaptation**: Use the Separate Information Maximization (SIM) objective function to further adapt the integrated model to the target domain.
5. **Verify effectiveness**:
- The paper verifies the effectiveness of the proposed method through a large number of experiments, proving that it has reached the current optimal level in both adaptation performance and transferability evaluation. In addition, this method also shows consistent effectiveness in various sub - settings, such as source models trained under different learning rates, batch sizes, optimizers, and pre - training weights.
### Summary
This paper mainly solves the following problems:
- How to fully utilize source models with different architectures in multi - source unsupervised domain adaptation.
- How to evaluate the transferability of these models without accessing target labels and source data.
- How to design an effective framework to select, integrate, and adapt these source models to improve the prediction performance in the target domain.
Through these improvements, the paper provides new ideas and methods for multi - source unsupervised domain adaptation and significantly improves the adaptation performance.