Granger Causality using Neural Networks

Malik Shahid Sultan,Samuel Horvath,Hernando Ombao

2024-08-07

Abstract:Dependence between nodes in a network is an important concept that pervades many areas including finance, politics, sociology, genomics and the brain sciences. One way to characterize dependence between components of a multivariate time series data is via Granger Causality (GC). Standard traditional approaches to GC estimation / inference commonly assume linear dynamics, however such simplification does not hold in many real-world applications where signals are inherently non-linear. In such cases, imposing linear models such as vector autoregressive (VAR) models can lead to mis-characterization of true Granger Causal interactions. To overcome this limitation, Tank et al (IEEE Transactions on Pattern Analysis and Machine Learning, 2022) proposed a solution that uses neural networks with sparse regularization penalties. The regularization encourages learnable weights to be sparse, which enables inference on GC. This paper overcomes the limitations of current methods by leveraging advances in machine learning and deep learning which have been demonstrated to learn hidden patterns in the data. We propose novel classes of models that can handle underlying non-linearity in a computationally efficient manner, simultaneously providing GC and lag order selection. Firstly, we present the Learned Kernel VAR (LeKVAR) model that learns kernel parameterized by a shared neural net followed by penalization on learnable weights to discover GC structure. Secondly, we show one can directly decouple lags and individual time series importance via decoupled penalties. This is important as we want to select the lag order during the process of GC estimation. This decoupling acts as a filtering and can be extended to any DL model including Multi-Layer Perceptrons (MLP), Recurrent Neural Networks (RNN), Long Short Term Memory Networks (LSTM), Transformers etc, for simultaneous GC estimation and lag selection.

Machine Learning

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

This paper attempts to address the limitations of linear assumptions when estimating Granger Causality (GC) in multivariate time series data. Traditional methods often assume linear dynamics, but in many real-world applications, signals are inherently nonlinear. This linear assumption can lead to misjudgment of the true Granger causality. To overcome this limitation, the authors propose a neural network-based approach that uses sparse regularization penalties to learn nonlinear dependency structures while simultaneously performing GC estimation and lag order selection.

### Main Contributions

1. **Learned Kernel Vector AutoRegressive (LeKV AR) Model**:
- A new model is proposed that does not require specifying any kernel function library but instead learns shared kernel functions through neural networks to learn nonlinear dependency structures from the data.

2. **Decoupling Lag and Time Series Components**:
- A simple and elegant method is proposed to measure the importance of each lag and time series component by decoupling them. This method includes applying sparse-inducing penalties separately to the time series components and lags to reduce overfitting and provide better GC estimation.

3. **Addressing Degenerate Objective Problem**:
- The paper identifies and addresses the issue of the objective function degenerating during model training by introducing weight normalization during the training process to improve model performance.

4. **Optimization Method**:
- The model is trained using standard deep learning optimizers (such as Adam) instead of traditional proximal gradient descent methods. This allows the model to support mini-batch training and use popular adaptive optimizers.

### Experimental Results

The paper conducts experiments on multiple benchmark datasets, including the VAR model, Lorenz-96 model, and simulated fMRI BOLD time series datasets. The experimental results show that the proposed LeKV AR model performs comparably or better than baseline methods, especially in handling nonlinear and data-scarce scenarios. Additionally, the LeKV AR model demonstrates excellent computational efficiency, being over 30 times faster than other models.

### Conclusion

The paper proposes a novel neural network-based method that effectively handles nonlinear dependency structures in multivariate time series data and achieves significant performance improvements in GC estimation and lag order selection. These methods have important potential value in real-world applications, particularly in fields requiring precise prediction and causal relationship analysis.