XTSFormer: Cross-Temporal-Scale Transformer for Irregular-Time Event Prediction in Clinical Applications

Tingsong Xiao,Zelin Xu,Wenchong He,Zhengkun Xiao,Yupu Zhang,Zibo Liu,Shigang Chen,My T. Thai,Jiang Bian,Parisa Rashidi,Zhe Jiang

2024-12-19

Abstract:Adverse clinical events related to unsafe care are among the top ten causes of death in the U.S. Accurate modeling and prediction of clinical events from electronic health records (EHRs) play a crucial role in patient safety enhancement. An example is modeling de facto care pathways that characterize common step-by-step plans for treatment or care. However, clinical event data pose several unique challenges, including the irregularity of time intervals between consecutive events, the existence of cycles, periodicity, multi-scale event interactions, and the high computational costs associated with long event sequences. Existing neural temporal point processes (TPPs) methods do not effectively capture the multi-scale nature of event interactions, which is common in many real-world clinical applications. To address these issues, we propose the cross-temporal-scale transformer (XTSFormer), specifically designed for irregularly timed event data. Our model consists of two vital components: a novel Feature-based Cycle-aware Time Positional Encoding (FCPE) that adeptly captures the cyclical nature of time, and a hierarchical multi-scale temporal attention mechanism, where different temporal scales are determined by a bottom-up clustering approach. Extensive experiments on several real-world EHR datasets show that our XTSFormer outperforms multiple baseline methods. The code is available at <a class="link-external link-https" href="https://github.com/spatialdatasciencegroup/XTSFormer" rel="external noopener nofollow">this https URL</a>.

Machine Learning,Artificial Intelligence

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve the problem of predicting clinical events with irregular time intervals in electronic health records (EHR). Specifically, the author focuses on how to accurately model and predict clinical events that occur in the medical environment in order to improve patient safety and the quality of care. The following are the key problems that this paper attempts to solve: 1. **Irregular time intervals**: - The time intervals of clinical event data are usually irregular, which makes it difficult for traditional time - series prediction methods (such as the standard Transformer model) to handle effectively. - For example, in the operating room, drug administration events may be recorded every few minutes, while in the pre - or post - operative stages, events may be recorded only every few hours or days. 2. **Periodicity and multi - scale effects**: - Clinical event sequences usually exhibit periodicity, periodicity and multi - scale interactions. For example, some drugs may be administered every 12 hours, while other drugs may be administered every two days. - These multi - scale patterns are very common in medical event data, but existing Neural Temporal Point Processes (Neural TPPs) methods cannot effectively capture these complex multi - scale interactions. 3. **High computational cost**: - Modeling complex patterns in long - time series, especially when the sequence length is long, will significantly increase the computational cost. - Existing methods may face a high computational burden when dealing with long event sequences. 4. **Limitations of existing methods**: - Traditional statistical TPP models can only capture simple event occurrence patterns, such as self - excitation phenomena. - Although RNN - based neural TPP models have solved the problem of vanishing gradients, they still face challenges when dealing with long - distance dependencies. - Transformer - based TPP models can capture long - distance dependencies, but they fail to fully consider the multi - scale patterns in event sequences. To solve these problems, the author proposes a new model named XTSFormer (Cross - Temporal - Scale Transformer). By introducing Feature - based Cycle - aware Time Positional Encoding (FCPE) and a cross - scale attention mechanism, this model can better handle clinical event data with irregular time intervals and capture its multi - scale and periodic characteristics.

XTSFormer: Cross-Temporal-Scale Transformer for Irregular-Time Event Prediction in Clinical Applications

Extracting Dynamic Information of Temporal Clinical Data to Predict the Outcome in Critically Ill Patients.

On Clinical Event Prediction in Patient Treatment Trajectory Using Longitudinal Electronic Health Records.

ExoTST: Exogenous-Aware Temporal Sequence Transformer for Time Series Prediction

DNA-T: Deformable Neighborhood Attention Transformer for Irregular Medical Time Series

Time-aware Heterogeneous Graph Transformer with Adaptive Attention Merging for Health Event Prediction

Hi-BEHRT: Hierarchical Transformer-based model for accurate prediction of clinical events using multimodal longitudinal electronic health records

Sparse-attentive meta temporal point process for clinical decision support

TrajGPT: Irregular Time-Series Representation Learning for Health Trajectory Analysis

Temporal Cross-Attention for Dynamic Embedding and Tokenization of Multimodal Electronic Health Records

Learning the Joint Representation of Heterogeneous Temporal Events for Clinical Endpoint Prediction

TEE4EHR: Transformer Event Encoder for Better Representation Learning in Electronic Health Records

Predicting Clinical Events Based on Raw Text: From Bag-of-Words to Attention-Based Transformers

Clinically-Inspired Multi-Agent Transformers for Disease Trajectory Forecasting from Multimodal Data

Temporal-spatial Correlation Attention Network for Clinical Data Analysis in Intensive Care Unit

Medformer: A Multi-Granularity Patching Transformer for Medical Time-Series Classification

DRFormer: Multi-Scale Transformer Utilizing Diverse Receptive Fields for Long Time-Series Forecasting

A Temporal Fusion Transformer for Long-term Explainable Prediction of Emergency Department Overcrowding

Feature extraction from unequal length heterogeneous EHR time series via dynamic time warping and tensor decomposition

Learning Hierarchical Representations of Electronic Health Records for Clinical Outcome Prediction.

Extrapolatable Transformer Pre-training for Ultra Long Time-Series Forecasting