Revisiting Network Perturbation for Semi-Supervised Semantic Segmentation

Sien Li,Tao Wang,Ruizhe Hu,Wenxi Liu

2024-11-08

Abstract:In semi-supervised semantic segmentation (SSS), weak-to-strong consistency regularization techniques are widely utilized in recent works, typically combined with input-level and feature-level perturbations. However, the integration between weak-to-strong consistency regularization and network perturbation has been relatively rare. We note several problems with existing network perturbations in SSS that may contribute to this phenomenon. By revisiting network perturbations, we introduce a new approach for network perturbation to expand the existing weak-to-strong consistency regularization for unlabeled data. Additionally, we present a volatile learning process for labeled data, which is uncommon in existing research. Building upon previous work that includes input-level and feature-level perturbations, we present MLPMatch (Multi-Level-Perturbation Match), an easy-to-implement and efficient framework for semi-supervised semantic segmentation. MLPMatch has been validated on the Pascal VOC and Cityscapes datasets, achieving state-of-the-art performance. Code is available from <a class="link-external link-https" href="https://github.com/LlistenL/MLPMatch" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition,Artificial Intelligence

What problem does this paper attempt to address?

### Problems Addressed by the Paper

In Semi-Supervised Semantic Segmentation (SSS), weak-to-strong consistency regularization techniques are widely applied to input-level and feature-level perturbations. However, the combination of network perturbations with weak-to-strong consistency regularization is relatively rare. This paper points out several issues with existing network perturbation methods in SSS, including:

1. **High computational cost**: Existing network perturbation methods typically require at least two networks, which is computationally expensive and more suitable for co-training frameworks.
2. **Difficulty in extracting diverse features**: These methods usually increase the possibility of overcoming cognitive biases caused by limited labeled data through supervision provided by networks with different initializations or architectures. However, they fail to extract sufficiently diverse features from the same image to perform weak-to-strong consistency regularization.
3. **Only applied to unlabeled data**: Existing network perturbation methods are only applied to unlabeled data, and it is unclear whether similar benefits can be obtained from labeled data.

To address these issues, this paper re-examines network perturbations and proposes a new network perturbation method—MLPMatch (Multi-Level-Perturbation Match), aimed at extending existing weak-to-strong consistency regularization methods. Additionally, the paper introduces a fluctuation learning process for labeled data, which is relatively rare in existing research. MLPMatch has been validated on the Pascal VOC and Cityscapes datasets, achieving state-of-the-art performance.