Diformer: A dynamic self-differential transformer for new energy power autoregressive prediction

Chengjie Zhou,Chao Che,Pengfei Wang,Qiang Zhang
DOI: https://doi.org/10.1016/j.knosys.2023.111061
IF: 8.139
2023-10-14
Knowledge-Based Systems
Abstract:Power prediction is important as a technical support mechanism in the global carbon neutrality initiative. Existing artificial intelligence methodologies frequently grapple with the challenge of excessive dependence on numerical space mapping. This reliance often results in sacrificing the precision of power prediction trends to achieve smoother prediction outcomes. Therefore, to address this issue, we propose a dynamic self-differential transformer (Diformer) tailored explicitly for univariate power time series forecasting to address this issue. Diformer incorporates two unique constructions: (i) a novel self-differential attention mechanism that enhances the importance of sequence trends through bidirectional difference features, resolving the issue of excessive reliance on token mapping in the numerical space in the transformer-based model during extracting features. (ii) a loss function called trade-loss, intensifying error loss by dynamically adjusting weights when the sequence trend changes. This formula mode significantly improves the model's ability to discern the direction of the sequence trend. Extensive experiments on six power time series datasets and publicly available power-related datasets demonstrate that Diformer significantly outperforms existing models and is better suited for power time series forecasting.
computer science, artificial intelligence
What problem does this paper attempt to address?