Abstract:In disentangled representation learning, the goal is to achieve a compact representation that consists of all interpretable generative factors in the observational data. Learning disentangled representations for graphs becomes increasingly important as graph data rapidly grows. Existing approaches often rely on Variational Auto-Encoder (VAE) or its causal structure learning-based refinement, which suffer from sub-optimality in VAEs due to the independence factor assumption and unavailability of concept labels, respectively. In this paper, we propose an unsupervised solution, dubbed concept-free causal disentanglement, built on a theoretically provable tight upper bound approximating the optimal factor. This results in an SCM-like causal structure modeling that directly learns concept structures from data. Based on this idea, we propose Concept-free Causal VGAE (CCVGAE) by incorporating a novel causal disentanglement layer into Variational Graph Auto-Encoder. Furthermore, we prove concept consistency under our concept-free causal disentanglement framework, hence employing it to enhance the meta-learning framework, called concept-free causal Meta-Graph (CC-Meta-Graph). We conduct extensive experiments to demonstrate the superiority of the proposed models: CCVGAE and CC-Meta-Graph, reaching up to $29\%$ and $11\%$ absolute improvements over baselines in terms of AUC, respectively.

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve This paper aims to address the issue of unsupervised causal disentangled representation learning in graph data. Specifically, the paper proposes a new method called Concept-free Causal Variational Graph Autoencoder (CCVGAE) and a meta-learning framework called Concept-free Causal Meta-Graph (CC-Meta-Graph). The core issues addressed are: 1. **Limitations of Existing Methods**: - Existing disentangled representation learning methods typically rely on Variational Autoencoders (VAE) or their improved versions for causal structure learning. These methods perform poorly under the independent factor assumption and even worse in the absence of concept labels. - Most VAE-based methods assume that the distribution in the latent space is an independent Gaussian distribution, leading to suboptimal solutions. 2. **Challenges in Unsupervised Settings**: - In unsupervised settings, there are no labels to guide the learning of generative factors, necessitating the development of new methods to learn causal structures directly from data. - It is essential to ensure that the learned generative factors are consistent across different graph data to be applicable to new data. 3. **Importance of Causal Disentanglement**: - The goal of disentangled representation learning is to extract all interpretable generative factors from observed data, which is particularly important in graph data due to their non-IID (non-Independent and Identically Distributed) and non-Euclidean characteristics. - Introducing causal structures can enhance disentanglement, especially when dealing with complex graph data. ### Main Contributions of the Paper 1. **Theoretical Analysis**: - The paper proves a compact upper bound for approximating optimal latent variable factors and proposes a causal disentanglement method that does not require concept labels. - Through linear causal modeling functions, it is possible to approximate optimal latent variable factors with high confidence. 2. **Methodological Innovations**: - The CCVGAE model is proposed, which introduces a new causal disentanglement layer in the Variational Graph Autoencoder (VGAE) to achieve unsupervised causal disentanglement. - The CC-Meta-Graph model is proposed, utilizing a meta-learning framework to transfer global information to new data, significantly reducing the need for new data. 3. **Experimental Validation**: - Extensive experiments on synthetic and real-world graph data validate the effectiveness of the proposed models in link prediction tasks, achieving absolute improvements of 29% and 11%, respectively. In summary, this paper addresses the limitations of existing methods in unsupervised settings by introducing a concept-free causal disentanglement approach, providing a new solution for disentangled representation learning in graph data.

Concept-free Causal Disentanglement with Variational Graph Auto-Encoder

Causal Flow-based Variational Auto-Encoder for Disentangled Causal Representation Learning

Structural Causality-based Generalizable Concept Discovery Models

Towards Human-like Perception: Learning Structural Causal Model in Heterogeneous Graph

Guided Variational Autoencoder for Disentanglement Learning

Learning Causally Disentangled Representations via the Principle of Independent Causal Mechanisms

CauF-VAE: Causal Disentangled Representation Learning with VAE and Causal Flows

Learning Network Representations with Disentangled Graph Auto-Encoder

SCM-VAE: Learning Identifiable Causal Representations Via Structural Knowledge.

Rethinking Controllable Variational Autoencoders

VACA: Designing Variational Graph Autoencoders for Causal Queries

CMVAE: Causal Meta VAE for Unsupervised Meta-Learning.

Disentanglement of Latent Representations via Causal Interventions

Unsupervised Belief Representation Learning with Information-Theoretic Variational Graph Auto-Encoders

Variational Disentangled Graph Auto-Encoders for Link Prediction

Causal Representation Learning Via Counterfactual Intervention

Causal disentanglement of multimodal data

Learning Disentangled Discrete Representations

Disentangled Representation Learning for Causal Inference with Instruments

Advanced Conditional Variational Autoencoders (A-CVAE): Towards interpreting open-domain conversation generation via disentangling latent feature representation

Cloud-VAE: Variational Autoencoder with Concepts Embedded