DuPL: Dual Student with Trustworthy Progressive Learning for Robust Weakly Supervised Semantic Segmentation

Yuanchen Wu,Xichen Ye,Kequan Yang,Jide Li,Xiaoqiang Li

2024-03-17

Abstract:Recently, One-stage Weakly Supervised Semantic Segmentation (WSSS) with image-level labels has gained increasing interest due to simplification over its cumbersome multi-stage counterpart. Limited by the inherent ambiguity of Class Activation Map (CAM), we observe that one-stage pipelines often encounter confirmation bias caused by incorrect CAM pseudo-labels, impairing their final segmentation performance. Although recent works discard many unreliable pseudo-labels to implicitly alleviate this issue, they fail to exploit sufficient supervision for their models. To this end, we propose a dual student framework with trustworthy progressive learning (DuPL). Specifically, we propose a dual student network with a discrepancy loss to yield diverse CAMs for each sub-net. The two sub-nets generate supervision for each other, mitigating the confirmation bias caused by learning their own incorrect pseudo-labels. In this process, we progressively introduce more trustworthy pseudo-labels to be involved in the supervision through dynamic threshold adjustment with an adaptive noise filtering strategy. Moreover, we believe that every pixel, even discarded from supervision due to its unreliability, is important for WSSS. Thus, we develop consistency regularization on these discarded regions, providing supervision of every pixel. Experiment results demonstrate the superiority of the proposed DuPL over the recent state-of-the-art alternatives on PASCAL VOC 2012 and MS COCO datasets. Code is available at

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper aims to address the confirmation bias issue in weakly supervised semantic segmentation (WSSS), especially when using image-level labels in the one-stage approach. Due to the inherent ambiguity of Class Activation Map (CAM), the one-stage pipeline is prone to confirmation bias caused by inaccurate CAM pseudo-labels during training, which hinders the improvement of final segmentation performance. Although recent work has alleviated this problem by filtering unreliable pseudo-labels with high thresholds, it leads to the lack of sufficient supervision for the model as many actually correct pseudo-labels are discarded.

To overcome the above limitations, the paper proposes a dual-student framework combined with trustworthy progressive learning (DuPL), which involves two sub-networks learning from each other to generate diverse CAMs and mitigate the confirmation bias caused by their own erroneous pseudo-labels. In addition, DuPL introduces a dynamic threshold adjustment strategy that allows more reliable pixels to participate in supervision, and adopts an adaptive noise filtering strategy to minimize noise in pseudo-labels. For those regions excluded from supervision due to unreliability, DuPL develops consistency regularization to ensure every pixel is properly utilized, thereby providing sufficient training. Experimental results demonstrate that DuPL outperforms current one-stage competitors on the PASCAL VOC 2012 and MS COCO datasets in terms of CAM pseudo-label quality and final segmentation performance, demonstrating its effectiveness in handling confirmation bias and fully utilizing pseudo-supervision.