Abstract:Multivariate time-series forecasting is vital in various domains, e.g., economic planning and weather prediction. Deep train-from-scratch models have exhibited effective performance yet require large amounts of data, which limits real-world applicability. Recently, researchers have leveraged the representation learning transferability of pre-trained Large Language Models (LLMs) to handle limited non-linguistic datasets effectively. However, incorporating LLMs with time-series data presents challenges of limited adaptation due to different compositions between time-series and linguistic data, and the inability to process multi-scale temporal information. To tackle these challenges, we propose LLM4TS, a framework for time-series forecasting with pre-trained LLMs. LLM4TS consists of a two-stage fine-tuning strategy: the \textit{time-series alignment} stage to align LLMs with the nuances of time-series data, and the \textit{forecasting fine-tuning} stage for downstream time-series forecasting tasks. Furthermore, our framework features a novel two-level aggregation method that integrates multi-scale temporal data within pre-trained LLMs, enhancing their ability to interpret time-specific information. In experiments across 7 time-series forecasting datasets, LLM4TS is superior to existing state-of-the-art methods compared with trained-from-scratch models in full-shot scenarios, and also achieves an average improvement of 6.84% in MSE in few-shot scenarios. In addition, evaluations compared with different self-supervised learning approaches highlight LLM4TS's effectiveness with representation learning in forecasting tasks.

What problem does this paper attempt to address?

### Problems Addressed by the Paper The paper "LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasters" aims to address several key issues in multivariate time-series forecasting: 1. **Data Efficiency**: Traditional deep learning models perform well in time-series forecasting but require a large amount of training data, which is often difficult to meet in practical applications. For example, in industrial manufacturing, sensor data from different products cannot be combined for analysis, resulting in limited data for each product type. 2. **Adaptability of Pre-Trained Language Models (LLMs) to Time-Series**: Although pre-trained large language models (LLMs) excel in natural language processing tasks and can be adapted to non-language datasets with minimal parameter fine-tuning, their application to time-series data faces two main challenges: - **Limited Adaptability to Time-Series Features**: Since LLMs' initial pre-training focuses on language corpora, they lack the ability to recognize key patterns and nuances in time-series data. - **Limited Capability to Handle Multi-Scale Temporal Information**: LLMs are good at understanding word order and context but perform poorly in handling multi-scale temporal information (such as seconds, minutes, hours, and specific dates). To address these issues, the authors propose the LLM4TS framework, which aligns pre-trained LLMs with time-series data through a two-stage fine-tuning strategy and introduces a novel two-layer aggregation method to integrate multi-scale temporal information. Experimental results show that LLM4TS outperforms existing state-of-the-art methods on multiple time-series forecasting benchmark datasets and excels in few-shot scenarios.

LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasters

Taming Pre-trained LLMs for Generalised Time Series Forecasting via Cross-modal Knowledge Distillation

Time-LLM: Time Series Forecasting by Reprogramming Large Language Models

Time Series Forecasting with LLMs: Understanding and Enhancing Model Capabilities

Multi-Patch Prediction: Adapting LLMs for Time Series Representation Learning

CALF: Aligning LLMs for Time Series Forecasting via Cross-modal Fine-Tuning

LLM-TS Integrator: Integrating LLM for Enhanced Time Series Modeling

Generalizable Memory-driven Transformer for Multivariate Long Sequence Time-series Forecasting

AutoTimes: Autoregressive Time Series Forecasters via Large Language Models

LeRet: Language-Empowered Retentive Network for Time Series Forecasting

Effective LSTMs with Seasonal-Trend Decomposition and Adaptive Learning and Niching-Based Backtracking Search Algorithm for Time Series Forecasting

Multi-Patch Prediction: Adapting Language Models for Time Series Representation Learning

An Evaluation of Standard Statistical Models and LLMs on Time Series Forecasting

LeMoLE: LLM-Enhanced Mixture of Linear Experts for Time Series Forecasting

ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoning

Temporal Data Meets LLM -- Explainable Financial Time Series Forecasting

Revisited Large Language Model for Time Series Analysis through Modality Alignment

Rethinking Time Series Forecasting with LLMs via Nearest Neighbor Contrastive Learning

GPT4MTS: Prompt-based Large Language Model for Multimodal Time-series Forecasting

Are Language Models Actually Useful for Time Series Forecasting?

Reprogramming Foundational Large Language Models(LLMs) for Enterprise Adoption for Spatio-Temporal Forecasting Applications: Unveiling a New Era in Copilot-Guided Cross-Modal Time Series Representation Learning