Abstract:Hawkes processes are often applied to model dependence and interaction phenomena in multivariate event data sets, such as neuronal spike trains, social interactions, and financial transactions. In the nonparametric setting, learning the temporal dependence structure of Hawkes processes is generally a computationally expensive task, all the more with Bayesian estimation methods. In particular, for generalised nonlinear Hawkes processes, Monte-Carlo Markov Chain methods applied to compute the doubly intractable posterior distribution are not scalable to high-dimensional processes in practice. Recently, efficient algorithms targeting a mean-field variational approximation of the posterior distribution have been proposed. In this work, we first unify existing variational Bayes approaches under a general nonparametric inference framework, and analyse the asymptotic properties of these methods under easily verifiable conditions on the prior, the variational class, and the nonlinear model. Secondly, we propose a novel sparsity-inducing procedure, and derive an adaptive mean-field variational algorithm for the popular sigmoid Hawkes processes. Our algorithm is parallelisable and therefore computationally efficient in high-dimensional setting. Through an extensive set of numerical simulations, we also demonstrate that our procedure is able to adapt to the dimensionality of the parameter of the Hawkes process, and is partially robust to some type of model mis-specification.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how to efficiently perform Bayesian non - parametric estimation in high - dimensional nonlinear Hawkes processes. Specifically, the paper focuses on using the variational Bayes method to infer the time - dependent structure of Hawkes processes on high - dimensional data sets. In particular, in the generalized nonlinear Hawkes processes, the traditional Monte Carlo Markov Chain (MCMC) method is difficult to apply due to high computational complexity.
### Background and Problem of the Paper
Hawkes processes are often used to model dependence and interaction phenomena in multivariate event data sets, such as neuronal spike trains, social interactions, and financial transactions. In the non - parametric setting, learning the time - dependent structure of Hawkes processes is usually a computationally expensive task, especially when using Bayesian estimation methods. For the generalized nonlinear Hawkes processes, the traditional MCMC method cannot be extended to high - dimensional processes in practical applications because it is necessary to calculate the doubly - intractable posterior distribution.
### Main Contributions
1. **Unifying Existing Methods**:
- The paper first unifies the existing variational Bayes methods under a general non - parametric inference framework and analyzes the asymptotically verifiable properties of these methods under prior, variational class, and nonlinear model conditions.
2. **Proposing a New Method**:
- Secondly, the paper proposes a new sparsity - inducing procedure and derives an adaptive mean - field variational algorithm for the popular Sigmoid Hawkes process. This algorithm is parallel, and thus computationally efficient in high - dimensional settings.
3. **Theoretical and Empirical Analysis**:
- The paper not only provides theoretical guarantees but also demonstrates the effectiveness of the algorithm through extensive numerical simulations. In particular, the algorithm can adapt to the dimension of the Hawkes process parameters and has a certain robustness to some types of model misspecification.
### Technical Details
- **Variational Bayes Framework**:
- The paper adopts the variational Bayes method to approximate the posterior distribution and finds the optimal variational posterior distribution by minimizing the Kullback - Leibler divergence.
- **Adaptive Two - Step Procedure**:
- Step 1: Calculate the variational Bayes posterior distribution using the complete graph model to obtain the L1 - norm estimates of each interaction function.
- Step 2: Estimate the connectivity graph parameter \(\delta\) by the threshold method, and then calculate the variational Bayes posterior distribution with \(\delta\) fixed.
- **Sigmoid Hawkes Process**:
- For the Sigmoid Hawkes process, the paper proposes a data - augmentation scheme that can efficiently calculate the mean - field approximate posterior distribution within the model.
### Conclusion
Through theoretical analysis and empirical research, the paper demonstrates the effectiveness and computational efficiency of the proposed adaptive variational Bayes method in high - dimensional nonlinear Hawkes processes. This method can not only adapt to the dimension of parameters but also partially deal with the problem of model misspecification.