Abstract:Statistical sampling, as we know it, works for single variables of the atmosphere. In one dimension (left panel), the distribution of reliable samples (blue curve) overlaps with the distribution of best guesses (black curve), and importantly, contains reality (red vertical line). In multidimensional spaces like resolved‐scale atmospheric dynamics (150 dimensions, right panel), sample points uniformly degrade the best estimate, so they do not contain reality. The attempt at capturing the evolution of the atmosphere with dynamically generated ensembles is futile. Uncertain quantities are often described through statistical samples. Can samples for numerical weather forecasts be generated dynamically? At a great expense, they can. With statistically constrained perturbations, a cloud of initial states is created and then integrated forward in time. By now, this technique has become ubiquitous in weather and climate research and operations. Ensembles are widely used, with demonstrated value. The atmosphere evolves in a multidimensional phase space. Does a cloud of ensemble solutions encompass the evolution of the real atmosphere? Theoretically, random perturbations in high‐dimensional spaces have negligible projection in any direction, including the error in the best estimate, therefore consistently degrading that. As the bulk of the perturbation variance lies in the null space of error, samples in multidimensional space do not contain reality. An evaluation suggests that initial and short‐range forecast error and ensemble perturbations are random draws from a high‐dimensional domain we call the subspace of possible error. Error in any initial condition is partly a result of stochastic observational and assimilation noise, while perturbations explore other, mostly independent directions from the subspace of possible error that may have resulted from other configurations of stochastic noise. What benefits may arise from the deterministic projection of such noise? Consistent with theoretical expectations, ensemble members consistently degrade the skill of the unperturbed forecast until medium range. The mean and all other products derived from ensembles suffer an 18‐hour loss in forecast Information. Since Information is a sufficient statistic, any rational user can benefit more from the unperturbed, than from an ensemble of weather forecasts. Furthermore, case‐dependent variations in the distribution or spread of ensembles have no impact on commonly used metrics. Can alternative, statistical applications provide comparable, or even higher‐quality probabilistic and other products, at the fraction of the cost of running an ensemble?

Combining parametric low‐dimensional ensemble postprocessing with reordering methods

Area-covering postprocessing of ensemble precipitation forecasts using topographical and seasonal conditions

Postprocessing of Ensemble Weather Forecasts Using Permutation-invariant Neural Networks

Ensemble weather forecast post-processing with a flexible probabilistic neural network approach

Multivariate post-processing of probabilistic sub-seasonal weather regime forecasts

Multivariate post‐processing of probabilistic sub‐seasonal weather regime forecasts

Generative machine learning methods for multivariate ensemble post-processing

From research to applications – Examples of operational ensemble post-processing in France using machine learning

Lead‐time‐continuous statistical postprocessing of ensemble weather forecasts

Evaluating Ensemble Post-Processing for Wind Power Forecasts

Sequential Aggregation of Probabilistic Forecasts -- Applicaton to Wind Speed Ensemble Forecasts

Machine learning methods for postprocessing ensemble forecasts of wind gusts: A systematic comparison

Ensemble solar forecasting using data-driven models with probabilistic post-processing through GAMLSS

Generation of scenarios from calibrated ensemble forecasts with a dual ensemble copula coupling approach

Ensemble forecasting: A foray of dynamics into the realm of statistics

Deep Learning for Post-Processing Ensemble Weather Forecasts

Post-processing multi-ensemble temperature and precipitation forecasts through an Exchangeable Gamma Normal model and its Tobit extension

Short-Range Numerical Weather Prediction Using Time-Lagged Ensembles

A Foray of Dynamics into the Realm of Statistics: A Review of Ensemble Forecasting

"The Simplest Idea One Can Have" for Seamless Forecasts with Postprocessing

Probabilistic Quantitative Precipitation Forecasting Using Ensemble Model Output Statistics