Abstract:Statistical sampling, as we know it, works for single variables of the atmosphere. In one dimension (left panel), the distribution of reliable samples (blue curve) overlaps with the distribution of best guesses (black curve), and importantly, contains reality (red vertical line). In multidimensional spaces like resolved‐scale atmospheric dynamics (150 dimensions, right panel), sample points uniformly degrade the best estimate, so they do not contain reality. The attempt at capturing the evolution of the atmosphere with dynamically generated ensembles is futile. Uncertain quantities are often described through statistical samples. Can samples for numerical weather forecasts be generated dynamically? At a great expense, they can. With statistically constrained perturbations, a cloud of initial states is created and then integrated forward in time. By now, this technique has become ubiquitous in weather and climate research and operations. Ensembles are widely used, with demonstrated value. The atmosphere evolves in a multidimensional phase space. Does a cloud of ensemble solutions encompass the evolution of the real atmosphere? Theoretically, random perturbations in high‐dimensional spaces have negligible projection in any direction, including the error in the best estimate, therefore consistently degrading that. As the bulk of the perturbation variance lies in the null space of error, samples in multidimensional space do not contain reality. An evaluation suggests that initial and short‐range forecast error and ensemble perturbations are random draws from a high‐dimensional domain we call the subspace of possible error. Error in any initial condition is partly a result of stochastic observational and assimilation noise, while perturbations explore other, mostly independent directions from the subspace of possible error that may have resulted from other configurations of stochastic noise. What benefits may arise from the deterministic projection of such noise? Consistent with theoretical expectations, ensemble members consistently degrade the skill of the unperturbed forecast until medium range. The mean and all other products derived from ensembles suffer an 18‐hour loss in forecast Information. Since Information is a sufficient statistic, any rational user can benefit more from the unperturbed, than from an ensemble of weather forecasts. Furthermore, case‐dependent variations in the distribution or spread of ensembles have no impact on commonly used metrics. Can alternative, statistical applications provide comparable, or even higher‐quality probabilistic and other products, at the fraction of the cost of running an ensemble?

Assessing the reliability of ensemble forecasting systems under serial dependence

Uniform reliability tests for forecasting systems with small lead time

Assessing the calibration of multivariate probabilistic forecasts

Testing for Pairwise Serial Independence Via the Empirical Distribution Function

Asymptotic Distribution Theory for Nonparametric Entropy Measures of Serial Dependence

Diagnostic Checking for the Adequacy of Nonlinear Time Series Models

A Two‐Stage Framework for Bias and Reliability Tests of Ensemble Hydroclimatic Forecasts

Unbiased evaluation and calibration of ensemble forecast anomalies

Generalized spectral tests for serial dependence

Consistent Testing for Serial Correlation of Unknown Form

Ensemble forecasting: A foray of dynamics into the realm of statistics

A hypothesis test for the long-term calibration in rating systems with overlapping time windows

Nonparametric Rank Test for Nonlinearity Detection

Regression Diagnostics meets Forecast Evaluation: Conditional Calibration, Reliability Diagrams, and Coefficient of Determination

On A Robust Test for Setar-Type Nonlinearity in Time Series Analysis

Ensemble size dependence of the logarithmic score for forecasts issued as multivariate normal distributions

A Foray of Dynamics into the Realm of Statistics: A Review of Ensemble Forecasting

On Ranking-based Tests of Independence

Testing independence in high dimensions with sums of rank correlations

RANK-BASED INDICES FOR TESTING INDEPENDENCE BETWEEN TWO HIGH-DIMENSIONAL VECTORS

Sequential Aggregation of Probabilistic Forecasts -- Applicaton to Wind Speed Ensemble Forecasts