Abstract:Recent advancements in multivariate time series forecasting have been propelled by Linear-based, Transformer-based, and Convolution-based models, with Transformer-based architectures gaining prominence for their efficacy in temporal and cross-channel mixing. More recently, Mamba, a state space model, has emerged with robust sequence and feature mixing capabilities. However, the suitability of the vanilla Mamba design for time series forecasting remains an open question, particularly due to its inadequate handling of cross-channel dependencies. Capturing cross-channel dependencies is critical in enhancing the performance of multivariate time series prediction. Recent findings show that self-attention excels in capturing cross-channel dependencies, whereas other simpler mechanisms, such as MLP, may degrade model performance. This is counterintuitive, as MLP, being a learnable architecture, should theoretically capture both correlations and irrelevances, potentially leading to neutral or improved performance. Diving into the self-attention mechanism, we attribute the observed degradation in MLP performance to its lack of data dependence and global receptive field, which result in MLP's lack of generalization ability. Based on the above insights, we introduce a refined Mamba variant tailored for time series forecasting. Our proposed model, \textbf{CMamba}, incorporates a modified Mamba (M-Mamba) module for temporal dependencies modeling, a global data-dependent MLP (GDD-MLP) to effectively capture cross-channel dependencies, and a Channel Mixup mechanism to mitigate overfitting. Comprehensive experiments conducted on seven real-world datasets demonstrate the efficacy of our model in improving forecasting performance.

Mamba4Cast: Efficient Zero-Shot Time Series Forecasting with State Space Models

A Mamba Foundation Model for Time Series Forecasting

Bi-Mamba4TS: Bidirectional Mamba for Time Series Forecasting

TimeMachine: A Time Series is Worth 4 Mambas for Long-term Forecasting

Bi-Mamba+: Bidirectional Mamba for Time Series Forecasting

UmambaTSF: A U-shaped Multi-Scale Long-Term Time Series Forecasting Method Using Mamba

FMamba: Mamba based on Fast-attention for Multivariate Time-series Forecasting

Is Mamba Effective for Time Series Forecasting?

MultiCast: Zero-Shot Multivariate Time Series Forecasting Using LLMs

C-Mamba: Channel Correlation Enhanced State Space Models for Multivariate Time Series Forecasting.

Integration of Mamba and Transformer -- MAT for Long-Short Range Time Series Forecasting with Application to Weather Dynamics

Valeo4Cast: A Modular Approach to End-to-End Forecasting

CMamba: Channel Correlation Enhanced State Space Models for Multivariate Time Series Forecasting

Integrating Mamba and Transformer for Long-Short Range Time Series Forecasting

MixMamba: Time Series Modeling with Adaptive Expertise

State-space models are accurate and efficient neural operators for dynamical systems

FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operators

Test Time Learning for Time Series Forecasting

SST: Multi-Scale Hybrid Mamba-Transformer Experts for Long-Short Range Time Series Forecasting

MambaStock: Selective state space model for stock prediction

Sequential Order-Robust Mamba for Time Series Forecasting