Abstract:Generative Flow Networks (GFlowNets, GFNs) are a generative framework for learning unnormalized probability mass functions over discrete spaces. Since their inception, GFlowNets have proven to be useful for learning generative models in applications where the majority of the discrete space is unvisited during training. This has inspired some to hypothesize that GFlowNets, when paired with deep neural networks (DNNs), have favourable generalization properties. In this work, we empirically verify some of the hypothesized mechanisms of generalization of GFlowNets. In particular, we find that the functions that GFlowNets learn to approximate have an implicit underlying structure which facilitate generalization. We also find that GFlowNets are sensitive to being trained offline and off-policy; however, the reward implicitly learned by GFlowNets is robust to changes in the training distribution.

What problem does this paper attempt to address?

### Problems the paper attempts to solve This paper aims to explore the generalization behavior of Generative Flow Networks (GFlowNets or GFNs) in unvisited state - space regions. Specifically, the researchers hope to empirically verify some hypothesized mechanisms regarding the generalization of GFlowNets and answer the following key questions: 1. **How do GFlowNets generalize in unvisited state - space regions?** - The researchers hope to understand whether GFlowNets can effectively allocate probability mass to regions not visited during training and explore the underlying mechanisms. 2. **What factors affect the generalization performance of GFlowNets?** - The researchers propose three main hypotheses: 1. **The influence of self - induced training distributions**: When GFlowNets sample from \( P_F(s' \mid s; \theta) \) or proportionally to \( R(s) \), the generalization effect is the best. 2. **The structure of the learning object**: The objects learned by GFlowNets have a structure, that is, \( P_F(s' \mid s) \) and \( F(s) \) are not arbitrary functions. 3. **The influence of reward complexity**: The generalization difficulty of GFlowNets is more affected by reward complexity (rather than reward distribution properties). 3. **How can these hypotheses be verified through experimental design?** - The researchers designed a series of benchmark tasks. By simplifying the assumptions to reduce the complexity and variables when training GFlowNets, specific variation factors are isolated to explore the generalization mechanism of GFlowNets. ### Main contributions 1. **Proposed a set of graph - generation benchmark tasks with different difficulties** for evaluating the generalization performance of GFlowNets. 2. **Verified some hypothesized characteristics regarding the generalization behavior of GFlowNets** and conducted empirical research through benchmark tasks. 3. **Identified and presented several observations and empirical findings** that provide a basis for understanding the generalization mechanism of GFlowNets. ### Experimental methods 1. **Distillation (regression) flow function**: By regressing the true flow function and the forward policy, the influence of GFlowNet training objectives and trajectory sampling is removed, the training process is simplified, and the learnability and generalization ability of the flow function are evaluated. 2. **Memory gap experiment**: By comparing the training performance on structured data and random unstructured data, the contribution of learning structured flow to generalization is evaluated. 3. **Offline and off - policy training**: By offline and off - policy training of GFlowNets on a known final - state data set, the influence of deviating from the self - induced training distribution on generalization is explored. ### Experimental results 1. **The difficulty of learning the forward policy \( P_F \) and the flow function \( F \) is similar**, and the model generalizes well on unvisited states, sometimes even better than the model trained online. 2. **The memory gap experiment shows that** learning structured flow can reduce the degree of memorization and is helpful for generalization. 3. **The offline and off - policy training experiment shows that** deviating from \( P_F(s' \mid s; \theta) \) has a certain negative impact on generalization, but this impact is relatively minor in some cases. ### Conclusion Through a series of empirical studies, this paper initially reveals some mechanisms of the generalization behavior of GFlowNets, providing an important reference for further understanding and optimizing GFlowNets.

Investigating Generalization Behaviours of Generative Flow Networks

On Generalization for Generative Flow Networks

Generative Flow Networks: a Markov Chain Perspective

Generative Flow Networks as Entropy-Regularized RL

DGFN: Double Generative Flow Networks

Bifurcated Generative Flow Networks

Stochastic Generative Flow Networks

A Theory of Non-Acyclic Generative Flow Networks

Pre-Training and Fine-Tuning Generative Flow Networks

DynGFN: Towards Bayesian Inference of Gene Regulatory Networks with GFlowNets

Bayesian Structure Learning with Generative Flow Networks

Local Search GFlowNets

Pessimistic Backward Policy for GFlowNets

Evolution Guided Generative Flow Networks

Generative Flow Networks for Precise Reward-Oriented Active Learning on Graphs

On Divergence Measures for Training GFlowNets

Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Network

Learning GFlowNets from partial episodes for improved convergence and stability

Better Training of GFlowNets with Local Credit and Incomplete Trajectories

BatchGFN: Generative Flow Networks for Batch Active Learning

GFlowNets for AI-Driven Scientific Discovery