Abstract:With the exponential expansion of e-commerce, an immense volume of historical sales data has been generated and amassed. This influx of data has created an opportunity for more accurate sales forecasting. While various sales forecasting methods and models have been applied in practice, existing ones often struggle to fully harness sales data and manage significant fluctuations. As a result, they frequently fail to make accurate predictions, falling short of meeting enterprise needs. Therefore, it is imperative to explore new models to enhance the accuracy and efficiency of sales forecasting. In this paper, we introduce a model tailored for sales forecasting based on a Transformer with encoder–decoder architecture and multi-head attention mechanisms. We have made specific modifications to the standard Transformer model, such as removing the Softmax layer in the last layer and adapting input embedding, position encoding, and feedforward network components to align with the unique characteristics of sales forecast data and the specific requirements of sales forecasting. The multi-head attention mechanism in our proposed model can directly compute the dot product results in a single step, addressing long-term time-dependent computation challenges while maintaining lower time complexity and greater interpretability. This enhancement significantly contributes to improving the model’s accuracy and efficiency. Furthermore, we provide a comprehensive formula representation of the model for the first time, facilitating better understanding and implementation. We conducted experiments using sales datasets that incorporate various factors influencing sales forecasts, such as seasons, holidays, and promotions. The results demonstrate that our proposed model significantly outperforms seven selected benchmark methods, reducing RMSLE, RMSWLE, NWRMSLE, and RMALE by approximately 48.2%, 48.5%, 45.2, and 63.0%, respectively. Additionally, ablation experiments on the multi-head attention and the number of encoder–decoders validate the rationality of our chosen model parameters.

Transformer-based power system energy prediction model

Transformers for Energy Forecast

Mat-Transformer-Based State Prediction Method for Information Equipment

A Transformer-Based Method of Multi-energy Load Forecasting in Integrated Energy System

A Comparative Study of Machine Learning-Based Time Series Prediction of Electricity Consumption in Virtual Power Plants

Forecasting Energy Consumption of a Public Building Using Transformer and Support Vector Regression

Small Sample Building Energy Consumption Prediction Using Contrastive Transformer Networks

Power Consumption Predicting and Anomaly Detection Based on Transformer and K-Means

Efficient Self-attention with Relative Position Encoding for Electric Power Load Forecasting

A novel Transformer-based network forecasting method for building cooling loads

Transformer-Based Model for Electrical Load Forecasting

Short-Term Electricity Consumption Forecasting Based on the EMD-Fbprophet-LSTM Method

A Power Forecasting Method for Ultra-Short-Term Photovoltaic Power Generation Using Transformer Model

Prediction method of intelligent building electricity consumption based on deep learning

Artificial intelligence-based power market price prediction in smart renewable energy systems: Combining prophet and transformer models

Energy consumption prediction using the GRU-MMattention-LightGBM model with features of Prophet decomposition

MAT-transformer-based state forecasting method for information devices

A Transformer-Based Multi-Entity Load Forecasting Method for Integrated Energy Systems

A Power Load Forecasting Method Based on Intelligent Data Analysis

Forecasting residential electricity consumption using the novel hybrid model

Achieving Sales Forecasting with Higher Accuracy and Efficiency: A New Model Based on Modified Transformer