Abstract:Exploring the missing values is an essential but challenging issue due to the complex latent spatio-temporal correlation and dynamic nature of time series. Owing to the outstanding performance in dealing with structure learning potentials, Graph Neural Networks (GNNs) and Recurrent Neural Networks (RNNs) are often used to capture such complex spatio-temporal features in multivariate time series. However, these data-driven models often fail to capture the essential spatio-temporal relationships when significant signal corruption occurs. Additionally, calculating the high-order neighbor nodes in these models is of high computational complexity. To address these problems, we propose a novel higher-order spatio-temporal physics-incorporated GNN (HSPGNN). Firstly, the dynamic Laplacian matrix can be obtained by the spatial attention mechanism. Then, the generic inhomogeneous partial differential equation (PDE) of physical dynamic systems is used to construct the dynamic higher-order spatio-temporal GNN to obtain the missing time series values. Moreover, we estimate the missing impact by Normalizing Flows (NF) to evaluate the importance of each node in the graph for better explainability. Experimental results on four benchmark datasets demonstrate the effectiveness of HSPGNN and the superior performance when combining various order neighbor nodes. Also, graph-like optical flow, dynamic graphs, and missing impact can be obtained naturally by HSPGNN, which provides better dynamic analysis and explanation than traditional data-driven models. Our code is available at <a class="link-external link-https" href="https://github.com/gorgen2020/HSPGNN" rel="external noopener nofollow">this https URL</a>.
What problem does this paper attempt to address?
This paper attempts to solve the problem of missing value imputation in multivariate time - series data, especially in the presence of complex spatio - temporal correlations and dynamic characteristics. Specifically, the paper focuses on the following key issues:
1. **Complex Spatio - Temporal Correlations**: Traditional methods based on Graph Neural Networks (GNNs) and Recurrent Neural Networks (RNNs) often struggle to capture complex spatio - temporal correlations when dealing with multivariate time - series data, especially when the signal is severely corrupted or the data missing rate is high.
2. **High Computational Complexity of Higher - Order Neighbor Node Computation**: Existing models face the problem of high computational complexity when computing higher - order neighbor nodes, which limits their efficiency and scalability in practical applications.
3. **Lack of Physical Mechanism Explanation**: Existing data - driven models usually lack a strong interpretability of the imputation mechanism. Especially when the data is severely missing, the performance of these models will be significantly affected.
To address these problems, the authors propose a new Higher - order Spatio - temporal Physics - incorporated Graph Neural Network (HSPGNN). The main contributions of HSPGNN include:
- **Combining Physical and Data - Driven Models**: By introducing a physical model with trainable physical parameters, HSPGNN can optimize the combination of different physical laws and provide better interpretability.
- **Reducing Computational Complexity**: Compared with traditional methods, HSPGNN can significantly reduce computational complexity. For example, for a time series of length \(M\), the traditional RNN has a spatial complexity of \(O(M^{2})\) and a time complexity of \(O(M^{3}N)\) when computing \((M - 1)\)-order derivatives, while HSPGNN can reduce the spatial complexity to approximately \(O(k_{t})\) and the time complexity to approximately \(O(k_{t}M^{2}N)\), where \(k_{t}\ll M\).
- **Experimental Verification**: Experimental results on four benchmark datasets show that HSPGNN is more effective and interpretable than existing methods in complex missing patterns, especially the multi - hop combination is better than the single - hop.
### Formula Summary
1. **Spatio - Temporal Attention Mechanism**:
- The dynamic Laplacian matrix \(L_{t - M:t}\) is obtained through the spatial attention mechanism.
2. **Higher - Order Partial Differential Equation (PDE)**:
- The general inhomogeneous PDE of the physical dynamic system is represented as:
\[
\frac{\partial u^{m}}{\partial t^{m}}=F(x, y, u_{t},\frac{\partial u_{t}}{\partial x},\frac{\partial u_{t}}{\partial y},\frac{\partial^{2}u_{t}}{\partial x^{2}},\frac{\partial^{2}u_{t}}{\partial y^{2}},\dots;\theta)+v_{t}(x, y)
\]
where \(u_{t}(x, y)\) is the observed physical value at position \((x, y)\) at time \(t\), \(F\) is the spatial function of \(u_{t}\), \(\theta\) is the parameter of \(F\), and \(v_{t}(x, y)\) is the unobservable external source term or perturbation.
3. **Discrete - Format Conversion**:
- Convert the above PDE into a discrete format, combined with the feature matrix \(X\) and represented as:
\[
\sum_{m = 1}^{M'}\lambda_{m}H^{m}X=\sum_{k = 0}^{K}\Theta_{k}XL^{k}+XW_{v}
\]
4. **Missing - Value Imputation Formula**:
- For partially observed values \(U = X\)