Towards Universal Large-Scale Foundational Model for Natural Gas Demand Forecasting

Xinxing Zhou,Jiaqi Ye,Shubao Zhao,Ming Jin,Zhaoxiang Hou,Chengyi Yang,Zengxiang Li,Yanlong Wen,Xiaojie Yuan

2024-09-24

Abstract:In the context of global energy strategy, accurate natural gas demand forecasting is crucial for ensuring efficient resource allocation and operational planning. Traditional forecasting methods struggle to cope with the growing complexity and variability of gas consumption patterns across diverse industries and commercial sectors. To address these challenges, we propose the first foundation model specifically tailored for natural gas demand forecasting. Foundation models, known for their ability to generalize across tasks and datasets, offer a robust solution to the limitations of traditional methods, such as the need for separate models for different customer segments and their limited generalization capabilities. Our approach leverages contrastive learning to improve prediction accuracy in real-world scenarios, particularly by tackling issues such as noise in historical consumption data and the potential misclassification of similar data samples, which can lead to degradation in the quaility of the representation and thus the accuracy of downstream forecasting tasks. By integrating advanced noise filtering techniques within the contrastive learning framework, our model enhances the quality of learned representations, leading to more accurate predictions. Furthermore, the model undergoes industry-specific fine-tuning during pretraining, enabling it to better capture the unique characteristics of gas consumption across various sectors. We conducted extensive experiments using a large-scale dataset from ENN Group, which includes data from over 10,000 industrial, commercial, and welfare-related customers across multiple regions. Our model outperformed existing state-of-the-art methods, demonstrating a relative improvement in MSE by 3.68\% and in MASE by 6.15\% compared to the best available model.

Machine Learning,Artificial Intelligence

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve

The paper aims to address several key issues in natural gas demand forecasting:

1. **Noise Data Handling**: Industrial datasets often contain a significant amount of noise, which may come from sensor errors, data recording issues, or irregular reporting intervals. Noise can obscure the true consumption patterns, making forecasting difficult.

2. **Differences in Consumption Patterns Across Industries**: The usage patterns of natural gas vary significantly across different industries, such as industrial, commercial, and residential sectors. This diversity and inconsistency make traditional forecasting methods challenging to apply.

To tackle the above challenges, the authors propose a general large-scale foundational model that leverages contrastive learning techniques to improve forecasting accuracy and employs a denoising module to filter out noise in the data. Additionally, the model undergoes industry-specific fine-tuning to better capture the unique characteristics of each industry. Experimental results show that the model outperforms existing methods on large-scale datasets, with a 3.68% improvement in Mean Squared Error (MSE) and a 6.15% improvement in Mean Absolute Scaled Error (MASE).