Abstract:Recent research has shown that large language models (LLMs) can be effectively used for real-world time series forecasting due to their strong natural language understanding capabilities. However, aligning time series into semantic spaces of LLMs comes with high computational costs and inference complexity, particularly for long-range time series generation. Building on recent advancements in using linear models for time series, this paper introduces an LLM-enhanced mixture of linear experts for precise and efficient time series forecasting. This approach involves developing a mixture of linear experts with multiple lookback lengths and a new multimodal fusion mechanism. The use of a mixture of linear experts is efficient due to its simplicity, while the multimodal fusion mechanism adaptively combines multiple linear experts based on the learned features of the text modality from pre-trained large language models. In experiments, we rethink the need to align time series to LLMs by existing time-series large language models and further discuss their efficiency and effectiveness in time series forecasting. Our experimental results show that the proposed LeMoLE model presents lower prediction errors and higher computational efficiency than existing LLM models.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to combine the advantages of large language models (LLMs) with the efficiency of linear models in time - series prediction to improve the prediction accuracy and computational efficiency. Specifically, the paper proposes a model named LeMoLE (LLM - enhanced mixture of linear experts), aiming to solve several key problems in existing methods in the following ways: 1. **Handling non - linear patterns**: Time - series data in the real world usually exhibit complex non - linear patterns, and traditional linear models are not effective in handling these non - linear relationships. LeMoLE can better capture these non - linear patterns by introducing multiple linear expert models with different look - back lengths. 2. **Handling long - range dependencies**: Linear models may face difficulties in handling long - range dependency relationships in time - series. LeMoLE effectively handles long - range dependencies by using multiple linear expert models, with each expert focusing on different time scales (short - term or long - term). 3. **Fusing multi - modal knowledge**: To further enhance the prediction ability of the model, LeMoLE introduces a multi - modal fusion mechanism, which uses pre - trained large language models to extract text features and combines them with time - series data. This not only improves the flexibility of the model but also enhances its ability to model complex time - series data. 4. **Improving computational efficiency**: Although existing time - series models based on LLMs perform well on certain tasks, they are usually computationally costly. LeMoLE achieves higher computational efficiency by optimizing the design of linear expert models and the multi - modal fusion mechanism. In summary, the main goal of LeMoLE is to develop a prediction model that can both maintain the efficiency of linear models and effectively handle complex time - series data. By combining the advantages of linear models and large language models, LeMoLE performs well in long - range prediction and few - sample learning scenarios.

LeMoLE: LLM-Enhanced Mixture of Linear Experts for Time Series Forecasting

Mixture-of-Linear-Experts for Long-term Time Series Forecasting

Time-LLM: Time Series Forecasting by Reprogramming Large Language Models

Time Series Forecasting with LLMs: Understanding and Enhancing Model Capabilities

Revisited Large Language Model for Time Series Analysis through Modality Alignment

From News to Forecast: Integrating Event Analysis in LLM-Based Time Series Forecasting with Reflection

LLM4TS: Aligning Pre-Trained LLMs as Data-Efficient Time-Series Forecasters

Temporal Data Meets LLM -- Explainable Financial Time Series Forecasting

An Evaluation of Standard Statistical Models and LLMs on Time Series Forecasting

LeRet: Language-Empowered Retentive Network for Time Series Forecasting

Toward Inference-optimal Mixture-of-Expert Large Language Models

A Comprehensive Evaluation of Large Language Models on Temporal Event Forecasting

FactorLLM: Factorizing Knowledge via Mixture of Experts for Large Language Models

OpenMoE: An Early Effort on Open Mixture-of-Experts Language Models

Time-MoE: Billion-Scale Time Series Foundation Models with Mixture of Experts

MoME: Mixture of Multimodal Experts for Generalist Multimodal Large Language Models

$\textbf{S}^2$IP-LLM: Semantic Space Informed Prompt Learning with LLM for Time Series Forecasting

Taming Pre-trained LLMs for Generalised Time Series Forecasting via Cross-modal Knowledge Distillation

Filtered not Mixed: Stochastic Filtering-Based Online Gating for Mixture of Large Language Models

AdaMoLE: Fine-Tuning Large Language Models with Adaptive Mixture of Low-Rank Adaptation Experts

A federated large language model for long-term time series forecasting