Sparse Graph Learning from Spatiotemporal Time Series

Andrea Cini,Daniele Zambon,Cesare Alippi
2023-08-02
Abstract:Outstanding achievements of graph neural networks for spatiotemporal time series analysis show that relational constraints introduce an effective inductive bias into neural forecasting architectures. Often, however, the relational information characterizing the underlying data-generating process is unavailable and the practitioner is left with the problem of inferring from data which relational graph to use in the subsequent processing stages. We propose novel, principled - yet practical - probabilistic score-based methods that learn the relational dependencies as distributions over graphs while maximizing end-to-end the performance at task. The proposed graph learning framework is based on consolidated variance reduction techniques for Monte Carlo score-based gradient estimation, is theoretically grounded, and, as we show, effective in practice. In this paper, we focus on the time series forecasting problem and show that, by tailoring the gradient estimators to the graph learning problem, we are able to achieve state-of-the-art performance while controlling the sparsity of the learned graph and the computational scalability. We empirically assess the effectiveness of the proposed method on synthetic and real-world benchmarks, showing that the proposed solution can be used as a stand-alone graph identification procedure as well as a graph learning component of an end-to-end forecasting architecture.
Machine Learning,Artificial Intelligence
What problem does this paper attempt to address?
The paper attempts to address the problem of inferring relational graphs from data in spatiotemporal time series analysis. Specifically: - **Background and Motivation**: In spatiotemporal time series analysis, Graph Neural Networks (GNNs) improve prediction performance by introducing relational constraints. However, in many real-world applications, the relational information in the underlying data generation process is often unknown, necessitating the inference of an appropriate relational graph from the data for subsequent processing. - **Objective**: Propose a novel probabilistic scoring method that can learn sparse graph structures from spatiotemporal time series data and maximize performance in prediction tasks. The method aims to control the sparsity of the learned graph and ensure computational scalability. The core contributions of the paper include: - Proposing a method based on score-function gradient estimators for end-to-end learning of probabilistic graphs in spatiotemporal data; - Designing a Monte Carlo estimator for the stochastic message-passing architecture; - Introducing two parameterized forms: a set of Bernoulli distributions and a set of edges sampled without replacement under sparsity constraints; - Proposing an efficient variance reduction method based on the Fréchet mean graph, which requires no additional parameter estimation and has low computational cost; - Proposing an approximate loss function to improve convergence speed. Through experimental validation, the method demonstrates excellent performance on spatiotemporal time series prediction benchmarks and enables sparse computation during both training and inference stages.