Analytic DAG Constraints for Differentiable DAG Learning

Zhen Zhang, Ignavier Ng, Dong Gong, Yuhang Liu, Mingming Gong, Biwei Huang, Kun Zhang, Anton van den Hengel, Javen Qinfeng Shi
2023-10-13
Abstract:Recovering underlying Directed Acyclic Graph (DAG) structures from observational data presents a formidable challenge due to the combinatorial nature of the DAG-constrained optimization problem. Recently, researchers have identified gradient vanishing as one of the primary obstacles in differentiable DAG learning and have proposed several DAG constraints to mitigate this issue. By developing the necessary theory to establish a connection between analytic functions and DAG constraints, we demonstrate that analytic functions from the set $\\{f(x) = c_0 + \sum_{i=1}c_ix^i|c_0 \geqslant 0; \forall i > 0, c_i > 0; r = \lim_{i\rightarrow \infty}c_{i}/c_{i+1} > 0\\}$ can be employed to formulate effective DAG constraints. Furthermore, we establish that this set of functions is closed under several functional operators, including differentiation, summation, and multiplication. Consequently, these operators can be leveraged to create novel DAG constraints based on existing ones. Additionally, we emphasize the significance of the convergence radius of an analytic function as a critical performance indicator. An infinite convergence radius is susceptible to gradient vanishing but less affected by nonconvexity. Conversely, a finite convergence radius aids in mitigating the gradient vanishing issue but may be more susceptible to nonconvexity. This property can be instrumental in selecting appropriate DAG constraints for various scenarios.
What problem does this paper attempt to address?