Abstract:Despite the impressive performance of recent unbiased Scene Graph Generation (SGG) methods, the current debiasing literature mainly focuses on the long-tailed distribution problem, whereas it overlooks another source of bias, i.e., semantic confusion, which makes the SGG model prone to yield false predictions for similar relationships. In this paper, we explore a debiasing procedure for the SGG task leveraging causal inference. Our central insight is that the Sparse Mechanism Shift (SMS) in causality allows independent intervention on multiple biases, thereby potentially preserving head category performance while pursuing the prediction of high-informative tail relationships. However, the noisy datasets lead to unobserved confounders for the SGG task, and thus the constructed causal models are always causal-insufficient to benefit from SMS. To remedy this, we propose Two-stage Causal Modeling (TsCM) for the SGG task, which takes the long-tailed distribution and semantic confusion as confounders to the Structural Causal Model (SCM) and then decouples the causal intervention into two stages. The first stage is causal representation learning, where we use a novel Population Loss (P-Loss) to intervene in the semantic confusion confounder. The second stage introduces the Adaptive Logit Adjustment (AL-Adjustment) to eliminate the long-tailed distribution confounder to complete causal calibration learning. These two stages are model agnostic and thus can be used in any SGG model that seeks unbiased predictions. Comprehensive experiments conducted on the popular SGG backbones and benchmarks show that our TsCM can achieve state-of-the-art performance in terms of mean recall rate. Furthermore, TsCM can maintain a higher recall rate than other debiasing methods, which indicates that our method can achieve a better tradeoff between head and tail relationships.

Causal Property Based Anti-conflict Modeling with Hybrid Data Augmentation for Unbiased Scene Graph Generation

PCPL: Predicate-Correlation Perception Learning for Unbiased Scene Graph Generation

Fast Contextual Scene Graph Generation with Unbiased Context Augmentation.

Attention Redirection Transformer with Semantic Oriented Learning for Unbiased Scene Graph Generation

Counterfactual Critic Multi-Agent Training for Scene Graph Generation

Scene Dynamics: Counterfactual Critic Multi-Agent Training for Scene Graph Generation.

Bridging Visual and Textual Semantics: Towards Consistency for Unbiased Scene Graph Generation

Debiased Scene Graph Generation for Dual Imbalance Learning

Probabilistic Modeling of Semantic Ambiguity for Scene Graph Generation

Prior Knowledge-driven Dynamic Scene Graph Generation with Causal Inference

Unbiased Scene Graph Generation via Two-stage Causal Modeling

Unbiased Scene Graph Generation in Videos

Unbiased Scene Graph Generation Using the Self-Distillation Method

Compositional Feature Augmentation for Unbiased Scene Graph Generation

Informative Scene Graph Generation via Debiasing

Learning to Generate an Unbiased Scene Graph by Using Attribute-Guided Predicate Features.

Unbiased Scene Graph Generation from Biased Training

Semantic Diversity-aware Prototype-based Learning for Unbiased Scene Graph Generation

Scene Graph Generation Method Based on External Information Guidance and Residual Scrambling

Generating Triples with Adversarial Networks for Scene Graph Construction

Relation-Specific Feature Augmentation for unbiased scene graph generation