Abstract:Time series forecasting (TSF) is a challenging problem in various real-world scenarios, such as industry, energy, weather, traffic, economics, and earthquake warning. TSF demands the model to have a high prediction accuracy. Despite the promising performance of deep learning-based methods in TSF tasks, mainstream forecasting models may sometimes produce results that deviate from the actual ground truth. Our analysis suggests that this may be attributed to the models’ limited ability to capture the frequency information that is abundantly present in real-world datasets. Currently, the Fourier Transform (FT) is the most widely used method for extracting frequency information, but it has some issues that lead to poor model performance, such as high-frequency noise caused by the Gibbs phenomenon and computational overhead of the inverse transformation in the FT-IFT process. To address these issues, we propose a novel frequency enhanced channel attention mechanism (FECAM) that models frequency interdependencies between channels based on Discrete Cosine Transform (DCT), which inherently mitigates the high-frequency noise caused by problematic periodicity during Fourier Transform. This approach improves the model’s capability to extract frequency features and resolves computational overhead concerns that arise from inverse transformations. Our contributions are threefold: (1) We propose a novel frequency enhanced channel attention mechanism that models frequency interdependencies between channels based on DCT, which improves the model’s capability to extract frequency features and resolves computational overhead concerns that arise from inverse transformations; (2) We theoretically prove that our method mitigates the Gibbs phenomenon, which introduces high frequency noise during Fourier Transform. We demonstrate that the result of 1D GAP linearly varies with the lowest frequency component of 1D DCT; (3) We demonstrate the generalization ability of the proposed method FECAM by embedding it into other networks, resulting in significant performance improvements when compared to the original model, with only a minor increase in parameters. Furthermore, we conduct extensive experiments on six different real-world TSF datasets to validate the effectiveness of our proposed model and compare it with several existing state-of-the-art models. Our findings indicate that the FECAM model is superior to these models in terms of accuracy, making it a promising solution for TSF in diverse real-world scenarios. Our codes and data are available at https://github.com/Zero-coder/FECAM.

TF(2)AN: A Temporal-Frequency Fusion Attention Network for Spectrum Energy Level Prediction

Boost Spectrum Prediction with Temporal-Frequency Fusion Network Via Transfer Learning

Adaptive Multi-Scale TF-net for High-Resolution Time-Frequency Representations

Hyperspectral and Multispectral Image Fusion Based on Deep Attention Network.

Fast Fourier Transform with Multihead Attention for Specific Emitter Identification

A Deep Long-Term Joint Temporal–Spectral Network for Spectrum Prediction

Spectral-Spatial-Temporal Attention Network for Hyperspectral Tracking.

Time-Frequency Attention for Monaural Speech Enhancement

FECAM: Frequency enhanced channel attention mechanism for time series forecasting

Speech Emotion Recognition via an Attentive Time-Frequency Neural Network

Spectral-Spatial Fused Attention Network for Hyperspectral Image Classification

SFANet: Spatial-Frequency Attention Network for Weather Forecasting

Anomaly Detection Method for Industrial Control System Operation Data Based on Time-Frequency Fusion Feature Attention Encoding

Spectrum Attention Mechanism for a Complex Neural Network

Frequency-Temporal Attention Network for Singing Melody Extraction

Enhanced spectrum prediction using deep learning models with multi-frequency supplementary inputs

FcaNet: Frequency Channel Attention Networks

A Deep-Learning-Based Method for Spectrum Sensing with Multiple Feature Combination

ATFNet: Adaptive Time-Frequency Ensembled Network for Long-term Time Series Forecasting

Fusionatt: Deep Fusional Attention Networks For Multi-Channel Biomedical Signals

HASTF: a hybrid attention spatio-temporal feature fusion network for EEG emotion recognition