A hybrid CNN-Transformer model for ozone concentration prediction

Yibin Chen,Xiaomin Chen,Ailan Xu,Qiang Sun,Xiaoyan Peng
DOI: https://doi.org/10.1007/s11869-022-01197-w
2022-04-23
Abstract:Ozone concentration has come to the fore as an important air quality indicator. However, ozone concentrations vary with meteorological conditions and the presence of other pollutants such as sulfur dioxide (SO2), nitrogen oxides (NOx), and carbon monoxide (CO). These relationships are nonlinear and dynamic, which makes it difficult for existing statistical and deep learning methods, e.g., autoregressive integrated moving average model (ARIMA), convolutional neural network (CNN), and long short-term memory network (LSTM) to fully capture the interaction between these factors and provide accurate prediction results. To solve this problem, we propose a hybrid model based on a CNN and a Transformer model called CNN-Transformer to predict the ozone concentration. CNN layers extract valuable information on feature dimensions, compensating for the Transformer encoder's limited ability to mine information from a multivariate dataset. Using multi-head attention layers between different encoder layers effectively improves the prediction accuracy, indicating that the information captured by the attention mechanism between global time series data effectively promotes the forecasting precision of our model. According to the data obtained from 14 monitoring stations in Beijing between 1 January 2014 and 31 July 2021, we take both meteorological factors, including wind speed, wind direction, minimum and maximum temperatures, and other environmental variables, i.e., NO, NO2, SO2, and CO into account as the predictors. Experimental results show that our proposed CNN-Transformer model outperforms other models, achieving excellent performance on both short-term forecast with an RMSE value of 7.75 and long-term forecast with an RMSE value of 16.27.
What problem does this paper attempt to address?