CAT: Exploiting Inter-Class Dynamics for Domain Adaptive Object Detection

Mikhail Kennerley,Jian-Gang Wang,Bharadwaj Veeravalli,Robby T. Tan

2024-03-28

Abstract:Domain adaptive object detection aims to adapt detection models to domains where annotated data is unavailable. Existing methods have been proposed to address the domain gap using the semi-supervised student-teacher framework. However, a fundamental issue arises from the class imbalance in the labelled training set, which can result in inaccurate pseudo-labels. The relationship between classes, especially where one class is a majority and the other minority, has a large impact on class bias. We propose Class-Aware Teacher (CAT) to address the class bias issue in the domain adaptation setting. In our work, we approximate the class relationships with our Inter-Class Relation module (ICRm) and exploit it to reduce the bias within the model. In this way, we are able to apply augmentations to highly related classes, both inter- and intra-domain, to boost the performance of minority classes while having minimal impact on majority classes. We further reduce the bias by implementing a class-relation weight to our classification loss. Experiments conducted on various datasets and ablation studies show that our method is able to address the class bias in the domain adaptation setting. On the Cityscapes to Foggy Cityscapes dataset, we attained a 52.5 mAP, a substantial improvement over the 51.2 mAP achieved by the state-of-the-art method.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### What problem does this paper attempt to solve?

This paper aims to address the **class imbalance problem in Domain Adaptive Object Detection (DAOD)**. Specifically, the paper proposes a method called **Class-Aware Teacher (CAT)** to tackle the following key issues:

1. **Class Imbalance**: In annotated datasets, the number of samples for certain classes is much higher than for others, leading to poor performance of the model on minority classes.
2. **Inaccurate Pseudo-Labels**: Due to class imbalance, the generated pseudo-labels may be inaccurate, which in turn affects the model's learning effectiveness.
3. **Inter-Class Relationships**: The similarity between different classes can lead to misclassification, especially where minority classes are easily misclassified as majority classes.

The paper introduces an **Inter-Class Relation module (ICRm)** to quantify the dynamic relationships between classes and uses this information for **Class-Relation Augmentation (CRA)** and **Inter-Class Loss (ICL)**, thereby improving the performance of minority classes in the target domain. Experimental results show that the CAT method achieves significant performance improvements on multiple benchmark datasets, particularly excelling in the **Cityscapes → Foggy Cityscapes** and **PASCAL VOC → Clipart1k** tasks.