Abstract:We develop a new methodology to solve the problem of identifying and selecting mesoscale weather patterns (MWPs) from high-dimensional spatio-temporal climate datasets. This problem is important and topical and has many implications for decision-makers across multiple sectors, such as urban design, urban climate, agriculture, transportation, energy, and disaster management. This problem involves selecting a small subset of data (specific days) from the original large dataset (decades long), such that it captures the essential information and characteristics of the large dataset, while minimizing redundancy. This is useful as it makes the dataset more manageable for processing, analysis, and insight gathering without degrading the overall quality of the information content. We develop a novel algorithm that is based on advanced machine learning and optimization techniques and consists of two stages: (1) spatial dimensionality reduction (SDR) to reduce the number of spatial cells analyzed while preserving as much relevant information as possible and (2) representative subset selection (RSS) to find a small subset of days in the dataset that captures the essential patterns, relationships, and information present in the full dataset—these are the mesoscale weather patterns . We demonstrate our methodology by applying it to a spatio-temporal dataset of atmospheric observations, the ERA5 dataset, in Singapore. The MWPs offer valuable insights into the region's diverse weather conditions and help researchers, climatologists, and policymakers comprehend the complex interactions between atmospheric elements.

Diffusion Maps for dimensionality reduction and visualization of meteorological data

Dynamic Mode Decomposition Analysis of Spatially Agglomerated Flow Databases

ADM: Adaptive Graph Diffusion for Meta-Dimension Reduction

Visual Analytics of the Spatio-temporal Multidimensional Air Monitoring Data

Dimensionality Reduction for Visual Data Mining of Earth Observation Archives

Manifold Diffusion Fields

Diffusion-Based Density-Equalizing Maps: an Interdisciplinary Approach to Visualizing Homicide Rates and Other Georeferenced Statistical Data

A Dashboard to Analysis and Synthesis of Dimensionality Reduction Methods in Remote Sensing

Debias Coarsely, Sample Conditionally: Statistical Downscaling through Optimal Transport and Probabilistic Diffusion Models

Selecting High-Dimensional Representations of Physical Systems by Reweighted Diffusion Maps

DYffusion: A Dynamics-informed Diffusion Model for Spatiotemporal Forecasting

DIRESA, a distance-preserving nonlinear dimension reduction technique based on regularized autoencoders

A Heat Diffusion Perspective on Geodesic Preserving Dimensionality Reduction

Functional Diffusion Maps

A Survey of Multivariate Spatial Data Visualization

A Geometric View of Data Complexity: Efficient Local Intrinsic Dimension Estimation with Diffusion Models

A data-driven framework for dimensionality reduction and causal inference in climate fields

DiffDA: a Diffusion Model for Weather-scale Data Assimilation

Visualization System of 3D Global Scale Meteorological Data

Methodology for Identifying Mesoscale Weather Patterns from High-Dimensional Climate Datasets

Spatial Downscaling of Sea Surface Temperature Using Diffusion Model