Omni-Dimensional Frequency Learner for General Time Series Analysis

Xianing Chen,Hanting Chen,Hailin Hu

2024-07-19

Abstract:Frequency domain representation of time series feature offers a concise representation for handling real-world time series data with inherent complexity and dynamic nature. However, current frequency-based methods with complex operations still fall short of state-of-the-art time domain methods for general time series analysis. In this work, we present Omni-Dimensional Frequency Learner (ODFL) model based on a in depth analysis among all the three aspects of the spectrum feature: channel redundancy property among the frequency dimension, the sparse and un-salient frequency energy distribution among the frequency dimension, and the semantic diversity among the variable dimension. Technically, our method is composed of a semantic-adaptive global filter with attention to the un-salient frequency bands and partial operation among the channel dimension. Empirical results show that ODFL achieves consistent state-of-the-art in five mainstream time series analysis tasks, including short- and long-term forecasting, imputation, classification, and anomaly detection, offering a promising foundation for time series analysis.

Computer Vision and Pattern Recognition,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: **How to elegantly extract time - series features in the frequency domain and extract the most informative parts from them, so as to achieve efficient processing of various time - series analysis tasks**.

Specifically, although current frequency - based methods can provide concise time - series representations, their performance on general time - series analysis tasks is still not as good as the state - of - the - art time - domain methods. Therefore, the authors propose the Omni - Dimensional Frequency Learner (ODFL) model, aiming to improve this situation by in - depth analysis of three key aspects of spectral features:

1. **Channel redundancy characteristics**: Frequency - domain features between different channels are highly similar.
2. **Energy distribution in the frequency dimension**: Frequency energy is mainly concentrated in the low - frequency part, but the high - frequency part also contains important trends.
3. **Semantic diversity in the variable dimension**: The model needs to adapt to the semantic information of different variables.

To achieve these goals, the authors introduce the following techniques:
- **Partial operation**: Apply a global filter only to some input channels to increase the feature diversity between channels.
- **Non - significant band feature extraction**: Focus on the significant frequency part while retaining the key historical information of other low - signal - to - noise - ratio parts.
- **Semantic adaptive filter**: Learn an adaptive filter to adapt to the semantic information of different variables.

Through these improvements, the ODFL model has achieved continuous state - of - the - art performance on five mainstream time - series analysis tasks (short - term and long - term prediction, imputation, classification, and anomaly detection), demonstrating its superiority and excellent task generalization ability.

### Formula summary

1. **Discrete Fourier Transform (DFT)**:
\[
X[k]=\sum_{n = 0}^{N - 1}x[n]e^{-j(2\pi/N)kn},\quad0\leq k\leq N - 1
\]
where \(j\) is the imaginary unit, and \(X[k]\) represents the spectrum of the sequence \(x[n]\) at the frequency \(\omega_k=\frac{2\pi k}{N}\).

2. **Inverse Discrete Fourier Transform (IDFT)**:
\[
x[n]=\frac{1}{N}\sum_{k = 0}^{N - 1}X[k]e^{j(2\pi/N)kn}
\]

3. **Global filter operation**:
\[
fX = X\odot K
\]
where \(K\in\mathbb{C}^{C\times D}\), and \(\odot\) represents element - wise multiplication.

4. **Partial channel operation**:
\[
fX_{1:[rp\times D]}=X_{1:[rp\times D]}\odot K
\]
where \(rp\) is the partial ratio (PR), and \(K\in\mathbb{C}^{[rp\times C]\times D}\).

5. **Significant band feature extraction**:
\[
fX_{1:[rp\times D][f_1,\dots,f_k]}=X_{1:[rp\times D][f_1,\dots,f_k]}\odot K
\]
where \(K\in\mathbb{C}^{[rp\times C]\times[rs\times D]}\), and \(rs = \frac{k}{D}\) is the sparsity rate (SR).

6. **Semantic adaptive filter**:
Learn an adaptive filter \(K\in\mathbb{C}^{[rp\times C]\times[rs\times D]}\), considering the real - part and imaginary - part mappings respectively.

Through these formulas and techniques, the ODFL model can dynamically extract the most informative parts in the frequency domain and convert them back to the time domain for further processing.