Abstract:Time series forecasting (TSF) is essential in various domains, and recent advancements in diffusion-based TSF models have shown considerable promise. However, these models typically adopt traditional diffusion patterns, treating TSF as a noise-based conditional generation task. This approach neglects the inherent continuous sequential nature of time series, leading to a fundamental misalignment between diffusion mechanisms and the TSF objective, thereby severely impairing performance. To bridge this misalignment, and inspired by the classic Auto-Regressive Moving Average (ARMA) theory, which views time series as continuous sequential progressions evolving from previous data points, we propose a novel Auto-Regressive Moving Diffusion (ARMD) model to first achieve the continuous sequential diffusion-based TSF. Unlike previous methods that start from white Gaussian noise, our model employs chain-based diffusion with priors, accurately modeling the evolution of time series and leveraging intermediate state information to improve forecasting accuracy and stability. Specifically, our approach reinterprets the diffusion process by considering future series as the initial state and historical series as the final state, with intermediate series generated using a sliding-based technique during the forward process. This design aligns the diffusion model's sampling procedure with the forecasting objective, resulting in an unconditional, continuous sequential diffusion TSF model. Extensive experiments conducted on seven widely used datasets demonstrate that our model achieves state-of-the-art performance, significantly outperforming existing diffusion-based TSF models. Our code is available on GitHub: <a class="link-external link-https" href="https://github.com/daxin007/ARMD" rel="external noopener nofollow">this https URL</a>.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is that in time - series forecasting (TSF) tasks, the existing diffusion - model - based methods fail to fully consider the continuous sequential characteristics of time series, which seriously affects the model performance. Specifically, traditional diffusion models usually handle time - series forecasting in the way of noise - conditional generation tasks, ignoring the inherent continuous sequential nature of time series, which causes a fundamental misalignment between the diffusion mechanism and the time - series forecasting objective. This misalignment not only affects the prediction accuracy of the model but also reduces the stability of the model.
To solve this problem, the paper proposes a new continuous - sequential diffusion model - Auto - Regressive Moving Diffusion (ARMD). This model draws on the classic Auto - Regressive Moving Average (ARMA) theory and believes that the time series is a continuous - sequential process evolving from previous data points. The ARMD model introduces prior information through the chained - diffusion method, accurately simulates the evolution process of the time series, and uses intermediate - state information to improve the prediction accuracy and stability.
### Specific Contributions
1. **Innovative Diffusion Mechanism**: Different from traditional methods that start from white Gaussian noise, the ARMD model adopts the chained - diffusion method, regards the future sequence as the initial state and the historical sequence as the final state, and generates intermediate sequences in the forward process through the sliding technique. This design makes the sampling process of the diffusion model consistent with the prediction objective, forming an unconditional, continuous - sequential diffusion TSF model.
2. **Utilization of Intermediate States**: The ARMD model reinterprets the diffusion process and generates intermediate states by sliding the series instead of generating them by adding noise as in traditional methods. This method eliminates the noise - related uncertainty, making the generation of intermediate states more deterministic, thereby improving the effect of model training.
3. **High - Performance Performance**: Through a large number of experiments on seven widely - used data sets, it is proved that the ARMD model has superior performance in multivariate time - series forecasting tasks and is significantly better than the existing diffusion - based time - series forecasting models. Especially on multiple data sets such as ETTm1, Stock, and Solar Energy, the ARMD model has achieved significant performance improvements in terms of mean - squared error (MSE) and mean - absolute error (MAE).
### Technical Details
- **Forward Diffusion (Evolution) Process**: ARMD takes the future sequence \(X_1^T\) as the initial state and the historical sequence \(X_{-T + 1}^0\) as the final state. The intermediate state \(X_{1 - t}^{T - t}\) is generated through the sliding operation, and the sliding at each step can be represented as:
\[
X_{1 - t}^{T - t}=\text{Slide}(X_{2 - t}^{T - t+ 1},1)
\]
This process is similar to the \(q\) process in the Denoising Diffusion Probability Model (DDPM).
- **Reverse Denoising (Reverse Evolution) Process**: ARMD uses the historical sequence \(X_{-T + 1}^0\) to gradually generate the future sequence \(X_1^T\). In each reverse step, the linear denoising network \(R(\cdot)\) predicts the evolution trend \(z_t\) to reverse - evolve \(X_{1 - t}^{T - t}\) to \(X_{2 - t}^{T - t+ 1}\). The prediction process can be represented as:
\[
\hat{X}_0(X_t,t,\theta)=W(t)\ast X_{1 - t}^{T - t}+(1 - bW(t))\ast D/(1 + cW(t))^d
\]
where \(W(t)\) is a weight coefficient that changes with the time step \(t\), and \(D\) is the distance prediction from the input \(X_{1 - t}^{T - t}\) to the target \(X_1^T\).
- **Sampling/Prediction Process**: In the sampling stage, ARMD starts from the historical sequence \(X_{-T + 1}^0\) and gradually generates the future sequence \(X_1^T\), making the sampling process aligned with the time - series forecasting objective.