Causal Intervention for Panoptic Scene Graph Generation

Shuang Liang,Long Zhang,Chi Xie,Lili Chen
DOI: https://doi.org/10.1109/icme57554.2024.10687509
2024-01-01
Abstract:Panoptic Scene Graph Generation (PSG) is a computer vision task that involves recognizing objects and their relationships within a given image. However, due to the long-tail distribution of the dataset, current PSG methods face bias issues. Previous debiasing methods usually rely on resampling and reweighting, which are unable to address the issue fundamentally, and may lead to underfitting for the head categories to some extent. Causal intervention, as a method to identify confounders from a causal inference perspective, can serve as a suitable means of debiasing. In this work, we propose a novel debiasing method based on causal intervention, which treats the long-tail distribution prior of the dataset as a confounder and eliminates it using the backdoor criterion. An uncertainty estimation module is further employed to assist in determining hard samples. Experiments demonstrate a significant improvement in the competitiveness of our approach compared to the baseline. Moreover, our method exhibits notable enhancements in accuracy and generalization on tail categories.
What problem does this paper attempt to address?