Is Mamba Effective for Time Series Forecasting?

Zihan Wang,Fanheng Kong,Shi Feng,Ming Wang,Xiaocui Yang,Han Zhao,Daling Wang,Yifei Zhang

2024-04-27

Abstract:In the realm of time series forecasting (TSF), it is imperative for models to adeptly discern and distill hidden patterns within historical time series data to forecast future states. Transformer-based models exhibit formidable efficacy in TSF, primarily attributed to their advantage in apprehending these patterns. However, the quadratic complexity of the Transformer leads to low computational efficiency and high costs, which somewhat hinders the deployment of the TSF model in real-world scenarios. Recently, Mamba, a selective state space model, has gained traction due to its ability to process dependencies in sequences while maintaining near-linear complexity. For TSF tasks, these characteristics enable Mamba to comprehend hidden patterns as the Transformer and reduce computational overhead compared to the Transformer. Therefore, we propose a Mamba-based model named Simple-Mamba (S-Mamba) for TSF. Specifically, we tokenize the time points of each variate autonomously via a linear layer. A bidirectional Mamba layer is utilized to extract inter-variate correlations and a Feed-Forward Network is set to learn temporal dependencies. Finally, the generation of forecast outcomes through a linear mapping layer. Experiments on thirteen public datasets prove that S-Mamba maintains low computational overhead and achieves leading performance. Furthermore, we conduct extensive experiments to explore Mamba's potential in TSF tasks. Our code is available at

Machine Learning

What problem does this paper attempt to address?

This paper mainly discusses the effectiveness of the Mamba model in Time Series Forecasting (TSF). Time series forecasting involves predicting future states using historical information, typically with two patterns: time dependence and inter-variable correlation. The Transformer model performs well in TSF, but its computational complexity grows quadratically with the number of variables and the length of observed sequences, resulting in low efficiency and high cost, limiting its deployment in practical applications. Mamba is a selective state space model that handles sequence dependencies with approximately linear computational complexity while maintaining efficiency. The paper proposes a new model based on Mamba called Simple-Mamba (S-Mamba) for time series forecasting. S-Mamba encodes the time points of each variable using linear layers, extracts inter-variable correlations with bidirectional Mamba layers, and learns time dependencies through a Feed-Forward Network. Finally, prediction results are generated through a linear mapping layer. Experiments show that S-Mamba not only has lower GPU memory requirements and training time on 13 publicly available datasets, but also outperforms existing state-of-the-art TSF models in terms of performance. Furthermore, the paper conducts extensive experiments to explore the potential of Mamba in TSF tasks. In summary, this paper addresses the problem of how to use the Mamba model to improve the efficiency and accuracy of time series forecasting while reducing the demand for computational resources. By proposing S-Mamba, the paper demonstrates the capability of Mamba in handling inter-variable correlations and time dependencies, providing a new effective method for time series forecasting.

Is Mamba Effective for Time Series Forecasting?

Foreformer: an Enhanced Transformer-Based Framework for Multivariate Time Series Forecasting

TFEformer: Temporal Feature Enhanced Transformer for Multivariate Time Series Forecasting

Bi-Mamba4TS: Bidirectional Mamba for Time Series Forecasting

Integrating Mamba and Transformer for Long-Short Range Time Series Forecasting

UmambaTSF: A U-shaped Multi-Scale Long-Term Time Series Forecasting Method Using Mamba

A Mamba Foundation Model for Time Series Forecasting

Bi-Mamba+: Bidirectional Mamba for Time Series Forecasting

FMamba: Mamba based on Fast-attention for Multivariate Time-series Forecasting

Mamba or Transformer for Time Series Forecasting? Mixture of Universals (MoU) Is All You Need

Integration of Mamba and Transformer -- MAT for Long-Short Range Time Series Forecasting with Application to Weather Dynamics

Hidformer: Hierarchical Dual-Tower Transformer Using Multi-Scale Mergence for Long-Term Time Series Forecasting

Enhancing Time Series Forecasting: A Hierarchical Transformer with Probabilistic Decomposition Representation

Simplified Mamba with Disentangled Dependency Encoding for Long-Term Time Series Forecasting

TCLN: A Transformer-based Conv-LSTM Network for Multivariate Time Series Forecasting

Sequential Order-Robust Mamba for Time Series Forecasting

SST: Multi-Scale Hybrid Mamba-Transformer Experts for Long-Short Range Time Series Forecasting

TimeMachine: A Time Series is Worth 4 Mambas for Long-term Forecasting

Generalizable Memory-driven Transformer for Multivariate Long Sequence Time-series Forecasting

C-Mamba: Channel Correlation Enhanced State Space Models for Multivariate Time Series Forecasting.

Multi-scale Transformer Pyramid Networks for Multivariate Time Series Forecasting