Abstract:Recently, multivariate time series forecasting tasks have garnered increasing attention due to their significant practical applications, leading to the emergence of various deep forecasting models. However, real-world time series exhibit pronounced non-stationary distribution characteristics. These characteristics are not solely limited to time-varying statistical properties highlighted by non-stationary Transformer but also encompass three key aspects: nested periodicity, absence of periodic distributions, and hysteresis among time variables. In this paper, we begin by validating this theory through wavelet analysis and propose the Transformer-based TwinS model, which consists of three modules to address the non-stationary periodic distributions: Wavelet Convolution, Period-Aware Attention, and Channel-Temporal Mixed MLP. Specifically, The Wavelet Convolution models nested periods by scaling the convolution kernel size like wavelet transform. The Period-Aware Attention guides attention computation by generating period relevance scores through a convolutional sub-network. The Channel-Temporal Mixed MLP captures the overall relationships between time series through channel-time mixing learning. TwinS achieves SOTA performance compared to mainstream TS models, with a maximum improvement in MSE of 25.8\% over PatchTST.
What problem does this paper attempt to address?
### Problems the paper attempts to solve
This paper aims to solve the non - stationarity problem in the Multivariate Time Series Forecasting (MTSF) task. Specifically, the authors focus on how to effectively model and handle the non - stationary distribution characteristics of time series in the real world. These characteristics include not only the time - varying statistical properties emphasized by the non - stationary Transformer, but also cover three key aspects:
1. **Nested periodicity**: The time series contains multiple nested and overlapping periods, and the periodic patterns and intensities at each time step are different.
2. **Absence of periodic distribution**: The time series shows different periodic patterns at specific stages or time periods, and some events may only occur within specific time periods.
3. **Lag effect**: There are similar periodic components among different time series, but they show a significant lag in the periodic distribution.
Although existing methods have solved the non - stationarity problem to a certain extent, they still cannot effectively decouple the nested periodic information, model the missing state of the periodic distribution, and capture the lag relationships between time series. Therefore, designing a model that can handle these problems is crucial for further improving the performance of the Transformer in the MTSF task.
### Main contributions
To meet the above challenges, the authors propose a new model named TwinS, which contains the following three key modules:
1. **Wavelet Convolution Module**: By simulating the wavelet transform technique, it extracts information from multiple nested periods during the initial embedding process.
2. **Periodic Aware Attention Module**: Combined with a convolution - based scoring sub - network, this mechanism can effectively model the non - stationary periodic distribution at different window scales, enabling TwinS to recognize and understand the complex non - stationary periodic patterns inherent in time series data.
3. **Channel - Temporal Mixer MLP Module**: Regarding the time series as a whole entity, it uses a Multi - Layer Perceptron (MLP) to capture the overall correlations between time variables.
Through these innovations, TwinS achieves superior performance over mainstream time series models on multiple popular datasets, with a maximum improvement of 25.8% in the MSE metric.
### Summary
The main purpose of this paper is to re - examine the MTSF task and propose a new method to solve non - stationarity problems such as nested periodicity, absence of periodic distribution, and lag effect in time series. By introducing the TwinS model, the authors demonstrate its effectiveness in handling these challenges and provide new perspectives and tools for future research.