Abstract:Foundation models of time series have not been fully developed due to the limited availability of time series corpora and the underexploration of scalable pre-training. Based on the similar sequential formulation of time series and natural language, increasing research demonstrates the feasibility of leveraging large language models (LLM) for time series. Nevertheless, the inherent autoregressive property and decoder-only architecture of LLMs have not been fully considered, resulting in insufficient utilization of LLM abilities. To fully revitalize the general-purpose token transition and multi-step generation capability of large language models, we propose AutoTimes to repurpose LLMs as autoregressive time series forecasters, which projects time series into the embedding space of language tokens and autoregressively generates future predictions with arbitrary lengths. Compatible with any decoder-only LLMs, the consequent forecaster exhibits the flexibility of the lookback length and scalability with larger LLMs. Further, we formulate time series as prompts, extending the context for prediction beyond the lookback window, termed in-context forecasting. By introducing LLM-embedded textual timestamps, AutoTimes can utilize chronological information to align multivariate time series. Empirically, AutoTimes achieves state-of-the-art with 0.1% trainable parameters and over $5\times$ training/inference speedup compared to advanced LLM-based forecasters. Code is available at this repository: <a class="link-external link-https" href="https://github.com/thuml/AutoTimes" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

### Problems Addressed by the Paper The paper "AutoTimes: An Autoregressive Time Series Predictor Using Large Language Models" aims to address several key issues present in existing time series forecasting methods: 1. **Deficiencies in Existing Time Series Base Models**: - **Dataset Limitations**: The lack of large-scale pre-training datasets hinders the development of foundational time series models. - **Technical Uncertainty**: The technical uncertainty of scalable backbone networks also affects the development of foundational time series models. 2. **Insufficient Application of Large Language Models (LLM) in Time Series Forecasting**: - **Limitations of Non-Autoregressive Methods**: Most existing LLM-based time series forecasting methods use non-autoregressive methods, leading to inconsistencies in model structure and generation methods, and failing to fully utilize the autoregressive properties of LLMs. - **Modal Differences**: Some methods use natural language as prompts, which may lead to modal differences and affect forecasting performance. 3. **Multi-Step Generation and Zero-Shot Generalization Capability**: - Existing time series forecasting methods perform poorly in multi-step generation and zero-shot generalization, which are crucial for practical applications. ### Solution To overcome the above issues, the paper proposes **AutoTimes**, a method for autoregressive time series forecasting using large language models. Specifically: 1. **Inheritance of Autoregressive Properties**: - **Maintaining Autoregressive Structure**: By freezing LLM parameters and introducing minimal additional parameters, AutoTimes maintains the autoregressive properties of LLMs, enabling predictions of arbitrary lengths. - **Flexible Backtracking Length**: This method is no longer limited by specific backtracking or prediction lengths, offering greater flexibility. 2. **Modal Alignment**: - **Time Series Tokenization**: Time series are segmented into continuous and non-overlapping fragments and embedded into the hidden space of LLMs. - **Positional Embedding**: Time stamps embedded by LLMs are used as positional embeddings to utilize time information and align multiple variables. 3. **Contextual Prediction**: - **Self-Prompting Mechanism**: By introducing relevant time series context, AutoTimes can perform contextual predictions, enhancing forecasting accuracy. - **Multimodal Utilization**: Combining information from other modalities further improves forecasting performance. ### Experimental Results Experimental results show that AutoTimes achieves state-of-the-art performance in multiple benchmark tests, saving over 80% in training and inference time, and demonstrating advantages in zero-shot generalization, contextual prediction, and scalability. ### Conclusion By proposing AutoTimes, the paper successfully addresses several key issues in existing time series forecasting methods, making significant progress in autoregressive properties, modal alignment, and multi-step generation. This method not only improves forecasting performance but also significantly reduces computational costs, offering broad application prospects.

AutoTimes: Autoregressive Time Series Forecasters via Large Language Models

Time-LLM: Time Series Forecasting by Reprogramming Large Language Models

Time Series Forecasting with LLMs: Understanding and Enhancing Model Capabilities

Taming Pre-trained LLMs for Generalised Time Series Forecasting via Cross-modal Knowledge Distillation

StockTime: A Time Series Specialized Large Language Model Architecture for Stock Price Prediction

Multi-Patch Prediction: Adapting Language Models for Time Series Representation Learning

Revisited Large Language Model for Time Series Analysis through Modality Alignment

An Evaluation of Standard Statistical Models and LLMs on Time Series Forecasting

LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasters

Large Language Models Are Zero-Shot Time Series Forecasters

Reprogramming Foundational Large Language Models(LLMs) for Enterprise Adoption for Spatio-Temporal Forecasting Applications: Unveiling a New Era in Copilot-Guided Cross-Modal Time Series Representation Learning

Effective LSTMs with Seasonal-Trend Decomposition and Adaptive Learning and Niching-Based Backtracking Search Algorithm for Time Series Forecasting

Position: What Can Large Language Models Tell Us about Time Series Analysis

TimeCMA: Towards LLM-Empowered Multivariate Time Series Forecasting via Cross-Modality Alignment

Generalizable Memory-driven Transformer for Multivariate Long Sequence Time-series Forecasting

Spatial-Temporal Large Language Model for Traffic Prediction

GPT4MTS: Prompt-based Large Language Model for Multimodal Time-series Forecasting

Multi-Patch Prediction: Adapting LLMs for Time Series Representation Learning

LeRet: Language-Empowered Retentive Network for Time Series Forecasting

From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflection

Empowering Time Series Analysis with Large Language Models: A Survey