Rigorous (in)validation of ecological models

Chuliang Song,Jonathan M. Levine

DOI: https://doi.org/10.1101/2024.09.19.613075

2024-09-23

Abstract:The complexity of ecosystems poses a formidable challenge for confidently invalidating ecological models, as current practices struggle to distinguish model inadequacies from the confounding effects of unobserved biotic or abiotic factors. The prevailing inability to falsify models has resulted in an accumulation of models but not an accumulation of confidence. Here, we introduce a new approach rooted in queueing theory, termed the covariance criteria, that establishes a rigorous test for model validity based on covariance relationships between observable quantities. These criteria set a high bar for models to pass by specifying necessary conditions that must hold regardless of unobserved factors. We demonstrate the broad applicability and discriminatory power of the covariance criteria by applying them to three long-standing challenges in ecological theory: resolving competing models of predator-prey functional responses, disentangling ecological and evolutionary dynamics in systems with rapid evolution, and detecting the often-elusive influence of higher-order species interactions. Across these diverse case studies, the covariance criteria consistently rule out inadequate models, while building strong confidence in those that provide strategically useful approximations. The covariance criteria approach is mathematically rigorous, computationally efficient, and often non-parametric, making it immediately applicable to existing data and models.

Ecology

What problem does this paper attempt to address?

The paper attempts to address the problem of how to confidently validate or falsify ecological models in ecology. Current methods struggle to distinguish between model inadequacies and the confounding effects of unobserved biotic or abiotic factors, leading to an inability to effectively verify the validity of models. To solve this issue, the authors introduce a new method based on cohort theory—the covariance criteria—to rigorously test the validity of models.

Specifically, the covariance criteria establish strict model validation standards through the covariance relationships between observed quantities. These criteria specify the necessary conditions that a model must meet, regardless of the presence of unobserved factors. The authors demonstrate the application of the covariance criteria in three long-standing ecological challenges: distinguishing competitive predator-prey functional response models, separating ecological and evolutionary dynamics in rapidly evolving systems, and detecting the effects of higher-order species interactions. Through these case studies, the covariance criteria effectively eliminate unsuitable models and enhance confidence in those that provide useful approximations. This method is mathematically rigorous, computationally efficient, and often non-parametric, making it directly applicable to existing data and models.