Deep Linear Hawkes Processes

Yuxin Chang,Alex Boyd,Cao Xiao,Taha Kass-Hout,Parminder Bhatia,Padhraic Smyth,Andrew Warrington

2024-12-27

Abstract:Marked temporal point processes (MTPPs) are used to model sequences of different types of events with irregular arrival times, with broad applications ranging from healthcare and social networks to finance. We address shortcomings in existing point process models by drawing connections between modern deep state-space models (SSMs) and linear Hawkes processes (LHPs), culminating in an MTPP that we call the deep linear Hawkes process (DLHP). The DLHP modifies the linear differential equations in deep SSMs to be stochastic jump differential equations, akin to LHPs. After discretizing, the resulting recurrence can be implemented efficiently using a parallel scan. This brings parallelism and linear scaling to MTPP models. This contrasts with attention-based MTPPs, which scale quadratically, and RNN-based MTPPs, which do not parallelize across the sequence length. We show empirically that DLHPs match or outperform existing models across a broad range of metrics on eight real-world datasets. Our proposed DLHP model is the first instance of the unique architectural capabilities of SSMs being leveraged to construct a new class of MTPP models.

Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the limitations of existing Marked Temporal Point Process (MTPP) models when dealing with different types of event sequences. Specifically, these problems include: 1. **Limited expressiveness**: Many existing MTPP models are unable to capture complex temporal dynamics and long - distance dependencies well. 2. **Low computational efficiency**: Although MTPP models based on the attention mechanism can be computed in parallel, their computational complexity grows quadratically with the sequence length; while MTPP models based on RNNs cannot be parallelized across sequence lengths, resulting in lower computational efficiency. 3. **Lack of support for irregularly - spaced observations**: Existing models have difficulty in efficiently processing time - series data with irregular intervals. To solve these problems, the author introduced a new MTPP model - the Deep Linear Hawkes Process (DLHP). By combining the advantages of modern deep state - space models (SSMs) and linear Hawkes processes (LHPs), DLHP achieves the following improvements: - **Parallelism and linear scaling**: DLHP can perform parallel computation while maintaining linear complexity, thereby significantly improving computational efficiency. - **Flexible intensity function**: DLHP can model the instantaneous rate of event occurrence more flexibly, thus better capturing complex temporal dynamics. - **Support for irregularly - spaced observations**: DLHP can gracefully handle time - series data with irregular intervals and is suitable for various application scenarios. Through these improvements, DLHP has demonstrated superior performance over existing models on multiple real - world datasets and performs better in handling longer sequences.

Deep Linear Hawkes Processes

Mamba Hawkes Process

Exploring Generative Neural Temporal Point Process

Multi-task Multi-Dimensional Hawkes Processes for Modeling Event Sequences

Decoupled Marked Temporal Point Process using Neural Ordinary Differential Equations

Long Horizon Forecasting With Temporal Point Processes

Dynamic Hawkes Processes for Discovering Time-evolving Communities' States behind Diffusion Processes

Disentangled Deep Multivariate Hawkes Process for Learning Event Sequences

Nonlinear Hawkes Processes in Time-Varying System

An Empirical Study: Extensive Deep Temporal Point Process

Neural Point Process for Learning Spatiotemporal Event Dynamics

Marked Neural Spatio-Temporal Point Process Involving a Dynamic Graph Neural Network

TPP-LLM: Modeling Temporal Point Processes by Efficiently Fine-Tuning Large Language Models

Marked Temporal Bayesian Flow Point Processes

Modeling and Applications for Temporal Point Processes

A Latent Hawkes Process Model for Event Clustering and Temporal Dynamics Learning with Applications in GitHub.

Deep graph kernel point processes

Intensity-free Integral-based Learning of Marked Temporal Point Processes

Linking Across Data Granularity: Fitting Multivariate Hawkes Processes to Partially Interval-Censored Data

Robust Deep Hawkes Process under Label Noise of Both Event and Occurrence

A Metric Space for Point Process Excitations