Abstract:Predicting future values in multivariate time series is vital across various domains. This work explores the use of large language models (LLMs) for this task. However, LLMs typically handle one-dimensional data. We introduce MultiCast, a zero-shot LLM-based approach for multivariate time series forecasting. It allows LLMs to receive multivariate time series as input, through three novel token multiplexing solutions that effectively reduce dimensionality while preserving key repetitive patterns. Additionally, a quantization scheme helps LLMs to better learn these patterns, while significantly reducing token use for practical applications. We showcase the performance of our approach in terms of RMSE and execution time against state-of-the-art approaches on three real-world datasets.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to use large - language models (LLMs) for zero - sample prediction of multivariate time series. Specifically, the paper proposes a method named MultiCast. This method enables LLMs to process multivariate time - series data through three novel token - reuse techniques, and at the same time reduces the token usage and improves the model performance through the SAX quantization scheme. The paper aims to explore how to use the powerful language representation ability of LLMs to capture patterns and dynamics in time series without additional training or fine - tuning, so as to achieve effective multivariate time - series prediction. ### Main contributions: 1. **Dimensional reuse techniques**: Three dimensional reuse techniques (digit - interleaving, value - interleaving, and value - concatenation) are proposed to compress multi - dimensional time - series data into a single string as the input of the LLM. 2. **SAX quantization**: The SAX quantization method is applied to pre - process the time series to reduce the computational cost and token usage, and at the same time help the LLM learn the patterns in the time series better. 3. **Experimental evaluation**: Experiments were carried out on three real - world data sets, comparing multiple traditional methods, machine - learning methods, and LLM - based methods, demonstrating the performance of MultiCast in multivariate time - series prediction. ### Key technical details: - **Dimensional reuse techniques**: - **Digit - Interleaving (DI)**: Interleave the numbers of each dimension at each time stamp. - **Value - Interleaving (VI)**: Arrange the values of each dimension at each time stamp in sequence. - **Value - Concatenation (VC)**: Connect the values of each dimension at each time stamp separated by commas. - **SAX quantization**: - **Symbolic Aggregate approXimation (SAX)**: Convert the time series into a symbolic representation, and reduce the data dimension and simplify the model input through piecewise aggregate approximation (PAA) and symbolization processing. ### Experimental results: - **Data sets**: - **Gas Rate**: Contains two - dimensional carbon dioxide emission data. - **Electricity**: Contains three - dimensional power measurement data. - **Weather**: Contains four - dimensional meteorological data. - **Performance evaluation**: - The root mean square error (RMSE) is used as the evaluation metric. - On multiple data sets, different variants of MultiCast are compared with traditional methods (such as ARIMA, LSTM) and other LLM - based methods (such as LLMTIME). - The results show that MultiCast performs better than or close to other methods in some dimensions, especially showing certain advantages when dealing with high - dimensional time series. ### Discussion: - **Challenges of multivariate time series**: The correlation between each dimension in multivariate time series increases the complexity of prediction, while traditional LLM methods usually only handle one - dimensional data. - **Advantages of zero - sample learning**: MultiCast avoids additional training or fine - tuning through zero - sample learning, improving the generalization ability of the model. - **Future work**: Further study the influence of different data sets and dimensional characteristics on the choice of reuse methods, and the possible performance improvement brought by using a larger - scale LLM (such as GPT - 4). In general, through introducing innovative dimensional reuse techniques and SAX quantization methods, this paper successfully solves the problem of using LLMs for zero - sample prediction of multivariate time series, and verifies its effectiveness on multiple real data sets.

MultiCast: Zero-Shot Multivariate Time Series Forecasting Using LLMs

LLM-Mixer: Multiscale Mixing in LLMs for Time Series Forecasting

Time Series Forecasting with LLMs: Understanding and Enhancing Model Capabilities

Mamba4Cast: Efficient Zero-Shot Time Series Forecasting with State Space Models

LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasters

Time-LLM: Time Series Forecasting by Reprogramming Large Language Models

AutoTimes: Autoregressive Time Series Forecasters via Large Language Models

Large Language Models Are Zero-Shot Time Series Forecasters

MixLinear: Extreme Low Resource Multivariate Time Series Forecasting with 0.1K Parameters

Generalizable Memory-driven Transformer for Multivariate Long Sequence Time-series Forecasting

TableTime: Reformulating Time Series Classification as Zero-Shot Table Understanding via Large Language Models

AutoCast++: Enhancing World Event Prediction with Zero-shot Ranking-based Context Retrieval

Taming Pre-trained LLMs for Generalised Time Series Forecasting via Cross-modal Knowledge Distillation

MM-Forecast: A Multimodal Approach to Temporal Event Forecasting with Large Language Models

Multi-Patch Prediction: Adapting LLMs for Time Series Representation Learning

xLSTM-Mixer: Multivariate Time Series Forecasting by Mixing via Scalar Memories

Tiny Time Mixers (TTMs): Fast Pre-trained Models for Enhanced Zero/Few-Shot Forecasting of Multivariate Time Series

Few-Shot Forecasting of Time-Series with Heterogeneous Channels

Multi-Patch Prediction: Adapting Language Models for Time Series Representation Learning

TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting