Abstract:We incorporate prior graph topology information into a Neural Controlled Differential Equation (NCDE) to predict the future states of a dynamical system defined on a graph. The informed NCDE infers the future dynamics at the vertices of simulated advection data on graph edges with a known causal graph, observed only at vertices during training. We investigate different positions in the model architecture to inform the NCDE with graph information and identify an outer position between hidden state and control as theoretically and empirically favorable. Our such informed NCDE requires fewer parameters to reach a lower Mean Absolute Error (MAE) compared to previous methods that do not incorporate additional graph topology information.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: How to integrate the known graph topology information into Neural Controlled Differential Equations (NCDE) to predict the future states of dynamic systems defined on graphs. Specifically, the author hopes to improve the performance of NCDE in processing spatio - temporal sequence data by introducing graph structure information, and in particular, when training on graph nodes, be able to more accurately infer the simulated convection data on graph edges.
### Specific description of the problem
1. **Background and motivation**:
- In many practical applications, such as traffic prediction, climate forecasting, etc., data often has spatio - temporal dependencies, and these dependencies can be represented by graph structures.
- Existing methods usually do not fully utilize the topological information of graphs, resulting in poor performance of models when dealing with sparse or partially missing time - series data.
2. **Research objectives**:
- Integrate the known graph topology information into NCDE to improve the prediction accuracy of future dynamic systems.
- Explore the effects of introducing graph information at different model architecture positions and find the optimal integration method.
3. **Key challenges**:
- How to effectively combine graph structure information with NCDE so that the model can achieve higher prediction accuracy while maintaining a lower number of parameters.
- It is necessary to verify the effectiveness of introducing graph information at different positions and select the optimal position and method.
### Solution
The author proposes Graph Neural Controlled Differential Equations (GNCDE) and solves the above problems in the following ways:
- **Introducing graph information**: By introducing the vertex transformation matrix \( A_{\text{outer}} \) at the outer position and the vertex adjacency matrix \( A_{\text{inner}} \) at the inner position, the graph structure information is integrated into NCDE.
- **Optimizing the model architecture**: Experiments show that introducing graph information at the outer position has a better effect, which can significantly reduce the number of model parameters and the Mean Absolute Error (MAE).
- **Application scenarios**: This method has potential application value in multiple fields, such as traffic flow prediction, river water level forecasting, climate and weather prediction, and disease spread modeling.
### Formula display
1. **Basic form of NCDE**:
\[
z(T) = z(0) + \int_0^T f_\theta(z(t)) \, dx(t)
\]
where \( z(t) \in \mathbb{R}^{d_z} \) is the hidden state, \( f_\theta: \mathbb{R}^{d_z} \to \mathbb{R}^{d_z \times d_x} \) is a vector field parameterized by a neural network, and \( x(t) \in \mathbb{R}^{d_x} \) is the control path.
2. **GNCDE after introducing graph information**:
\[
Z(T) = Z(0) + \int_0^T g_\gamma(Z(t)) A_{\text{outer}} \, dH(t)
\]
where \( Z(t) \in \mathbb{R}^{|V| \times d_z} \) is the hidden state containing spatial information, \( H(t) \in \mathbb{R}^{|V| \times d_h} \) is the time - hidden state, and \( A_{\text{outer}} \) is the vertex transformation matrix.
Through these improvements, the author shows how to use graph structure information to improve the prediction performance of NCDE, especially when dealing with complex spatio - temporal dependent data.