Abstract:Recent normalization-based methods have shown great success in tackling the distribution shift issue, facilitating non-stationary time series forecasting. Since these methods operate in the time domain, they may fail to fully capture the dynamic patterns that are more apparent in the frequency domain, leading to suboptimal results. This paper first theoretically analyzes how normalization methods affect frequency components. We prove that the current normalization methods that operate in the time domain uniformly scale non-zero frequencies, and thus, they struggle to determine components that contribute to more robust forecasting. Therefore, we propose FredNormer, which observes datasets from a frequency perspective and adaptively up-weights the key frequency components. To this end, FredNormer consists of two components: a statistical metric that normalizes the input samples based on their frequency stability and a learnable weighting layer that adjusts stability and introduces sample-specific variations. Notably, FredNormer is a plug-and-play module, which does not compromise the efficiency compared to existing normalization methods. Extensive experiments show that FredNormer improves the averaged MSE of backbone forecasting models by 33.3% and 55.3% on the ETTm2 dataset. Compared to the baseline normalization methods, FredNormer achieves 18 top-1 results and 6 top-2 results out of 28 settings.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is that in non - stationary time - series prediction, existing normalization - based methods, when operating in the time domain, are unable to fully capture dynamic patterns in the frequency domain, resulting in sub - optimal prediction results. Specifically, when normalizing in the time domain, existing methods will uniformly scale all non - zero frequency components, which makes it difficult for them to determine the frequency components that contribute to more robust predictions. Therefore, the paper proposes a new method - FredNormer, which improves the prediction performance of non - stationary time - series by observing the data set from a frequency perspective and adaptively increasing the weights of key frequency components.
### Main Problems
1. **Limitations of Time - Domain Normalization**:
- Existing normalization methods mainly operate in the time domain, which may lead to the inability to fully capture dynamic patterns in time - series, especially those patterns that are more obvious in the frequency domain.
- These methods uniformly scale all frequency components in the time domain, and thus may overlook frequency features that contribute significantly to prediction.
2. **Requirement for Frequency - Domain Modeling**:
- A method is needed that can better model the dynamic characteristics of time - series in the frequency domain, thereby improving the accuracy of prediction.
- The specific goal is to suppress non - stationarity by learning time - invariant frequency components (i.e., stable frequencies) to achieve more robust prediction.
### Solution
The paper proposes FredNormer, a frequency - domain normalization module, which mainly consists of the following two components:
1. **Frequency Stability Measurement**:
- Calculate the statistical stability of each frequency component, defined as the ratio of the mean to the standard deviation of the frequency amplitude (i.e., the reciprocal of the coefficient of variation).
- Through this measurement, stable and unstable frequency components can be identified.
2. **Frequency Stability Weighting Layer**:
- Use linear projection to adjust the frequency stability measurement, introduce sample - specific variations, and increase distribution diversity.
- Convert the adjusted frequency spectrum back to the time domain as the input for subsequent prediction models.
### Experimental Results
The experimental results show that FredNormer significantly improves the performance of prediction models on multiple data sets. Especially on data sets with complex frequency characteristics, such as ETTm2, it improves the average MSE of PatchTST and iTransformer by 33.3% and 55.3% respectively. In addition, FredNormer achieves excellent results of 18 first - places and 6 second - places in 28 settings, significantly outperforming the baseline normalization methods RevIN and SAN.
### Summary
Through theoretical analysis and experiments, the paper proves that existing time - domain normalization methods have limitations when dealing with non - stationary time - series, and FredNormer effectively solves these problems by adaptively increasing the weights of key frequency components in the frequency domain, significantly improving the performance of prediction models.