Abstract:Graphical flows add further structure to normalizing flows by encoding non-trivial variable dependencies. Previous graphical flow models have focused primarily on a single flow direction: the normalizing direction for density estimation, or the generative direction for inference. However, to use a single flow to perform tasks in both directions, the model must exhibit stable and efficient flow inversion. This work introduces graphical residual flows, a graphical flow based on invertible residual networks. Our approach to incorporating dependency information in the flow, means that we are able to calculate the Jacobian determinant of these flows exactly. Our experiments confirm that graphical residual flows provide stable and accurate inversion that is also more time-efficient than alternative flows with similar task performance. Furthermore, our model provides performance competitive with other graphical flows for both density estimation and inference tasks.
What problem does this paper attempt to address?
### Problems the paper attempts to solve
This paper aims to solve the stability and efficiency problems of existing graphical flows when performing bidirectional tasks. Specifically:
1. **Unidirectionality limitation**: Existing graphical flow models mainly focus on tasks in a single direction, for example:
- **Normalizing direction**: Used for density estimation.
- **Generative direction**: Used for inference.
2. **Lack of stable bidirectional transformation**: In order to perform bidirectional tasks (such as density estimation and inference) simultaneously in one flow, the model needs to have a stable and efficient bidirectional transformation ability. However, existing methods do not emphasize this point.
3. **Numerical stability problem**: Theoretically, normalizing flows (NFs) are reversible, but in practical applications, if the Lipschitz constant of the reverse flow transformation is too large, numerical errors may be amplified, resulting in an unstable reverse transformation.
4. **Insufficient encoding of dependency structure**: Although some methods (such as residual flows) obtain stable reverse transformations through Lipschitz constraints, they do not encode the dependency structure of the target distribution.
### Proposed solutions
To solve the above problems, the paper proposes **Graphical Residual Flows (GRF)**, a graphical flow model based on invertible residual networks. The main features of GRF include:
- **Encoding of dependency structure**: By masking the weight matrix, the dependency information in the Bayesian network (BN) is encoded into the flow. This enables each variable to obtain information from its ancestor nodes, similar to the information propagation method in the message - passing algorithm.
- **Stable and accurate reverse transformation**: By constraining the Lipschitz bound of the flow transformation, the stability and accuracy of the reverse transformation are ensured.
- **Accurate calculation of the Jacobian determinant**: Since the dependency structure is encoded, the value of the Jacobian determinant can be accurately calculated without approximation.
### Experimental verification
The author verifies the performance of GRF on density estimation and inference tasks through experiments and compares it with other existing methods. The experimental results show that GRF is not only competitive in performance, but also performs better in terms of the stability and efficiency of the reverse transformation.
In conclusion, this paper proposes a new graphical flow model - Graphical Residual Flows (GRF), which can ensure the performance of bidirectional tasks while providing more stable and efficient reverse transformations, and is suitable for scenarios requiring reliable bidirectional operations.