Abstract:Daily internet communication relies heavily on tree-structured graphs, embodied by popular data formats such as XML and JSON. However, many recent generative (probabilistic) models utilize neural networks to learn a probability distribution over undirected cyclic graphs. This assumption of a generic graph structure brings various computational challenges, and, more importantly, the presence of non-linearities in neural networks does not permit tractable probabilistic inference. We address these problems by proposing sum-product-set networks, an extension of probabilistic circuits from unstructured tensor data to tree-structured graph data. To this end, we use random finite sets to reflect a variable number of nodes and edges in the graph and to allow for exact and efficient inference. We demonstrate that our tractable model performs comparably to various intractable models based on neural networks.

What problem does this paper attempt to address?

The core problem that this paper attempts to solve is to design a probability model, especially a generative probability model, for tree - structured graph data. Existing generative models (such as neural - network - based models) are usually used for undirected cyclic graphs and face computational challenges when dealing with these graphs. In particular, due to the nonlinear characteristics in neural networks, exact probability inference cannot be carried out. Moreover, when existing models are applied to tree - structured graphs, they fail to fully utilize their hierarchical structure characteristics, thus increasing unnecessary computational costs. ### Specific problem description 1. **Limitations of existing models**: - Existing generative models (such as neural - network - based models) are mainly used for undirected cyclic graphs, assuming a general graph structure, which brings various computational challenges. - The nonlinear characteristics in neural networks make probability inference infeasible and require reliance on approximation techniques. 2. **Characteristics of tree - structured graphs**: - Tree - structured graphs are widely present in many scientific fields (such as chemoinformatics, physics, and natural language processing) and are represented in formats such as JSON and XML. - These data structures have hierarchical relationships and are suitable for describing objects and their relationships. 3. **Requirements for interpretability and efficiency**: - In sensitive application areas (such as healthcare, finance, and network security), it is crucial to provide fast and non - approximate decisions. - Probability circuits (PCs) are guaranteed to be able to answer complex probability queries efficiently, but are currently mainly applied to fixed - size unstructured data (such as tensors). ### Goals of the paper The paper proposes a new probability model - Sum - Product - Set Networks (SPSNs), aiming to expand the application range of probability circuits from fixed - size unstructured data to variable - size tree - structured graph data. Specific goals include: 1. **Design a probability model suitable for tree - structured graphs**: SPSNs handle variable - size tree - structured graph data by introducing set units, ensuring that the model can handle different numbers of nodes and edges and allowing for exact and efficient inference. 2. **Ensure the interpretability and efficiency of the model**: SPSNs are traceable under standard structural constraints and can perform exact probability inference in polynomial time without the need for approximation techniques. 3. **Verify the effectiveness of the model**: Through experimental verification, SPSNs can be comparable in performance to complex and intractable neural - network - based models while providing more accurate probability inference capabilities. ### Summary By introducing Sum - Product - Set Networks (SPSNs), this paper solves the computational challenges and nonlinear problems faced by existing generative models when dealing with tree - structured graphs, providing an efficient and interpretable probability model.

Sum-Product-Set Networks: Deep Tractable Models for Tree-Structured Graphs

Tractable Probabilistic Graph Representation Learning with Graph-Induced Sum-Product Networks

GraphSPNs: Sum-Product Networks Benefit From Canonical Orderings

The Sum-Product Theorem: A Foundation for Learning Tractable Models

Interventional Sum-Product Networks: Causal Inference with Tractable Probabilistic Models

Top-Down Bayesian Posterior Sampling for Sum-Product Networks

Deep Compression of Sum-Product Networks on Tensor Networks

On Neural Networks as Infinite Tree-Structured Probabilistic Graphical Models

Maximum A Posteriori Inference in Sum-Product Networks

A survey of sum–product networks structural learning

Modeling Graphs Using a Mixture of Kronecker Models

Tree tensor networks for generative modeling

Stochastic Solutions for Dense Subgraph Discovery in Multilayer Networks

Some Might Say All You Need Is Sum

Polynomial-time derivation of optimal k-tree topology from Markov networks

Generating Likely Counterfactuals Using Sum-Product Networks

On the Expressive Power of Tree-Structured Probabilistic Circuits

Probabilistic graphs using coupled random variables

Generalizing Tree Probability Estimation Via Bayesian Networks

ARTree: A Deep Autoregressive Model for Phylogenetic Inference