Abstract:In data analysis and forecasting, particularly for multivariate long-term time series, challenges persist. The Transformer model in deep learning methods has shown significant potential in time series forecasting. The Transformer model's dot-product attention mechanism, however, due to its quadratic computational complexity, impairs training and forecasting efficiency. In addition, the Transformer architecture has limitations in modeling local features and dealing with multivariate cross-dimensional dependency relationship. In this article, a Multi-Scale Convolution Enhanced Transformer model (MSCformer) is proposed for multivariate long-term time series forecasting. As an alternative to modeling the time series in its entirety, a segmentation strategy is designed to convert the input original series into segmented forms with different lengths, then process time series segments using a new constructed multi-Dependency Aggregation module. This multi-Scale segmentation approach reduces the computational complexity of the attention mechanism part in subsequent models, and for each segment of length corresponds to a specific time scale, it also ensures that each segment retains the semantic information of the data sequence level, thereby comprehensively utilizing the multi-scale information of the data while more accurately capturing the real dependency of the time series. The Multi-Dependence Aggregate module captures both cross-temporal and cross-dimensional dependencies of multivariate long-term time series and compensates for local dependencies within the segments thereby captures local series features comprehensively and addressing the issue of insufficient information utilization. MSCformer synthesizes dependency information extracted from various temporal segments at different scales and reconstructs future series using linear layers. MSCformer exhibits higher forecasting accuracy, outperforming existing methods in multiple domains including energy, transportation, weather, electricity, disease and finance.

MICN: Multi-scale Local and Global Context Modeling for Long-term Series Forecasting

Hidformer: Hierarchical Dual-Tower Transformer Using Multi-Scale Mergence for Long-Term Time Series Forecasting

Multi-scale convolution enhanced transformer for multivariate long-term time series forecasting

TCLN: A Transformer-based Conv-LSTM Network for Multivariate Time Series Forecasting

Spatial-Temporal Convolutional Transformer Network for Multivariate Time Series Forecasting

Multi-Scale Dilated Convolution Network for Long-Term Time Series Forecasting

MP3Net:Multi-scale Patch Parallel Prediction Networks for Multivariate Time Series Forecasting

A Multiscale Interactive Recurrent Network for Time-Series Forecasting

Multivariate Temporal Convolutional Network: A Deep Neural Networks Approach for Multivariate Time Series Forecasting

GCformer: An Efficient Framework for Accurate and Scalable Long-Term Multivariate Time Series Forecasting

MSGNet: Learning Multi-Scale Inter-Series Correlations for Multivariate Time Series Forecasting

MDCNet: Long-term Time Series Forecasting with Mode Decomposition and 2D Convolution

GCformer: an Efficient Solution for Accurate and Scalable Long-Term Multivariate Time Series Forecasting

Multi-Task Spatial-Temporal Transformer for Multi-Variable Meteorological Forecasting

CNformer: a convolutional transformer with decomposition for long-term multivariate time series forecasting

Multi-Scale Convolutional Neural Network with Time-Cognition for Multi-Step Short-Term Load Forecasting

Adaptive Convolutional Forecasting Network Based on Time Series Feature-Driven

Adapt to Small-Scale and Long-Term Time Series Forecasting with Enhanced Multidimensional Correlation

Towards Better Forecasting by Fusing Near and Distant Future Visions

FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series Classification

Learning the Dynamic Correlations and Mitigating Noise by Hierarchical Convolution for Long-term Sequence Forecasting