GeoFormer: A Vision and Sequence Transformer-based Approach for Greenhouse Gas Monitoring

Madhav Khirwar, Ankur Narang

2024-02-12

Abstract:Air pollution represents a pivotal environmental challenge globally, playing a major role in climate change via greenhouse gas emissions and negatively affecting the health of billions. However predicting the spatial and temporal patterns of pollutants remains challenging. The scarcity of ground-based monitoring facilities and the dependency of air pollution modeling on comprehensive datasets, often inaccessible for numerous areas, complicate this issue. In this work, we introduce GeoFormer, a compact model that combines a vision transformer module with a highly efficient time-series transformer module to predict surface-level nitrogen dioxide (NO2) concentrations from Sentinel-5P satellite imagery. We train the proposed model to predict surface-level NO2 measurements using a dataset we constructed with Sentinel-5P images of ground-level monitoring stations, and their corresponding NO2 concentration readings. The proposed model attains high accuracy (MAE 5.65), demonstrating the efficacy of combining vision and time-series transformer architectures to harness satellite-derived data for enhanced GHG emission insights, proving instrumental in advancing climate change monitoring and emission regulation efforts globally.

Machine Learning,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem this paper attempts to address is the accurate prediction of ground-level nitrogen dioxide (NO₂) concentrations to monitor greenhouse gas emissions and support global climate change monitoring and emission control. Specifically, the paper proposes a new model called GeoFormer, which combines Vision Transformer modules and efficient Time-series Transformer modules to predict ground-level NO₂ concentrations using Sentinel-5P satellite images.

The main contributions of the paper include:
1. **Dataset Construction**: Introducing a comprehensive dataset that includes high-resolution Sentinel-5P satellite images and corresponding ground-level NO₂ concentration measurements, with daily granularity and a time span of 15 months.
2. **Compact and Efficient Spatiotemporal Transformer Model**: Proposing a new transformer-based model that effectively leverages the spatial and temporal dynamics of NO₂ emissions, thereby surpassing existing methods in terms of accuracy while significantly reducing the required computational resources.

Through this approach, the paper demonstrates the effectiveness of combining satellite data with deep learning techniques, providing a new solution for environmental monitoring.